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ABSTRACT 


Many  production  facilities  rely  on  the  intuition  of  the 
quality  assurance  inspector  for  determining  what  will  be 
visually  inspected  and  for  what  length  of  time  on  a  finished 
product.  In  this  thesis  a  model  for  visual  inspection  of  a 
finished  product  is  developed.  The  key  parameters  in  the 
model  are  the  probability  of  worker  error,  the  probability  of 
inspector  error,  and  the  cost  of  system  error.  Paired 
comparisons  of  error  phenomena  from  operational  personnel  are 
converted  to  probabilities  using  the  indirect  numerical 
estimation  technique. 

The  model  is  used  in  a  goal  program  to  optimize  the  use 
of  inspector  time  in  a  production  facility.  The  model  and  the 
goal  program  are  applied  to  a  military  problem  to  demonstrate 
the  broad  applicability  of  the  methodology.  The  military 
problem  is  the  allocation  of  inspection  time  before  firing  an 


artillery  weapon  to  insure  accurate  and  timely  delivery  of 


projectiles 
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INTRODUCTION 


I . 


A.  GENERAL 

Assume  that  all  the  resources  needed  for  a  particular 
production  process  are  already  purchased.  They  represent  sunk 
costs  to  the  production  facility.  The  only  cost  of  doing 
business  then  is  the  cost  incurred  by  producing  unacceptable 
goods.  If  finished  goods  are  flawed  or  late  when  delivered 
to  the  consumer  in  a  market  economy  then  the  producer 
eventually  loses  market  share.  Continued  loss  of  market  share 
leads  to  annihilation  of  the  production  business.  Thus,  the 
decision  maker's  task  is  to  allocate  the  available  resources 
to  achieve  timely  delivery  of  error-free  products.  This  is 
the  situation  in  which  some  mid-level  managers  find 
themselves.  In  other  words,  given  little  or  no  control  over 
the  resources  on  hand,  they  are  to  perform  as  well  as  possible 
but  are  not  to  fail. 

The  typical  response  to  the  mid-level  manager's  production 
problem  is  to  employ  part  of  the  allotted  labor  resources  as 
quality  assurance  inspectors.  Use  of  labor  resources  as 
inspectors  rather  than  producers  decreases  production.  The 
time  required  to  conduct  the  inspections  slows  the  remaining 
production.  The  use  of  the  inspectors,  however,  decreases  the 
number  of  flaws  in  the  finished  product.  The  manager  faces 
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a  tradeoff  between  two  favorable  objectives:  better  finished 
products  versus  more  finished  products. 

Assume  that  all  worker  errors  in  the  production  process 
that  can  lead  to  a  flawed  finished  product  can  be  identified. 
Assume  that  each  worker  error^  (where  i  =  l,...,m)  is 
independent  and  mutually  exclusive  of  each  other.  Therefore, 
given  that  a  flawed  product  is  found  bearing  the  effects  of 
one  error  from  the  production  process: 

(1-1)  2  P  ( error,  )  =  1.0 

i  =  l 

Assume  a  cost  distribution  can  be  identified  for  error. 
From  those  distributions  assume  that  mean  costs  can  be 
derived.  Now  the  expected  cost  of  doing  business  (EC)  per 
each  item  produced  can  be  written: 

(1-2)  L  n  (errorj  *  C  (errorj  =  EC 

where  m  is  the  number  of  independent  errors 
P  is  "probability" 

C  is  "mean  cost" 

The  probability  for  a  flawed  finished  product  due  to  error 
on  task1  [  P  ( error1 )  ]  is  the  probability  that  both  the 

worker  erred  on  the  task  and  the  inspector  failed  to  identify 
the  worker’s  error.  The  inspector's  ability  to  positively 
identify  flawed  products  as  being  unacceptable  increases  over 
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time.  Naturally,  the  cost  of  doing  business  can  be  reduced 
if  the  allocation  of  available  inspector  time  can  be  driven 
toward  errors  that  occur  more  frequently  and  which  have 
greater  cost.  How  should  the  available  inspection  time  be 
divided  between  error-prone  tasks  to  best  reduce  the  cost  of 
doing  business?  The  focus  of  this  thesis  is  how  to  allocate 
the  available  inspection  time  to  best  accomplish  the  manager’s 
multiple  objectives  of  speed  and  thoroughness. 

Goal  programming  is  a  convenient  tool  for  simultaneously 
satisfying  several  objectives.  The  first  half  of  this  thesis 
proposes  a  methodology  for  allocating  inspection  time  based 
on  goal  programming.  The  second  half  uses  the  artillery  gun 
chief's  inspection  problem  to  illustrate  the  methodology. 

In  offering  a  solution  to  the  allocation  of  inspection 
time  problem,  the  thesis  uses  several  timely,  efficient,  and 
sound  techniques  to  derive  a  deterministic  probability  for 
worker  failure  on  each  error-prone  task,  the  expected  time 
that  an  inspector  spends  inspecting  each  task,  and  the  cost 
distribution  of  error  on  each  task.  The  hope  is  that  these 
techniques  are  sufficiently  convenient  that  they  can  be 
readily  adopted  by  mid-level  managers  in  their  allocation  of 
inspection  resource  problems. 
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B.  DEFINITIONS 


Some  terminology  is  peculiar  to  the  study  of  human 
reliability.  Other  terminology  is  peculiar  to  the  U.S.  Army 
and  the  field  artillery.  Appendix  F  contains  a  glossary  of 
those  terms . 

C.  ORGANIZATION  OF  THE  THESIS 

Chapter  2  presents  the  experimental  results  and  theories 
that  are  the  underpinnings  of  the  proposed  methodology  for 
solving  the  allocation  of  inspection  time  problem.  The 
chapter  documents  alternative  techniques  by  which  error  rates 
and  time  estimates  may  be  derived  other  than  by  empirical 
observation.  The  chapter  reviews  a  technique  for  estimating 
costs  of  errors  and  discusses  the  attributes  of  goal 
programming . 

Chapters  3  through  5  present  the  problem  that  illustrates 
the  proposed  methodology.  Chapter  3,  Background  to  the 
Artillery  Problem,  describes  the  allocation  of  inspection  time 
problem  as  it  applies  to  the  gun  section  and  the  artillery 
fire  support  system.  The  discussion  of  artillery  explains  the 
tasks  of  the  various  crewmen  and  the  errors  they  make. 
Chapter  4  develops  a  model  for  the  artillery  gun  chief's 
problem.  Chapter  5  applies  the  methodology  to  an  active  army 
unit  at  Fort  Ord,  California.  Chapter  6,  Conclusions  and 
Recommendations,  proposes  other  applications  of  this 
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methodology  and  suggests  areas  for  future  research.  The 
appendices  give  the  output  from  the  computer  runs  discussed 
in  Chapter  5,  the  surveys  of  respondents,  and  step-by-step 
procedures  for  computing  the  solution  to  the  artillery  gun 
chief's  inspection  problem. 
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II.  DEVELOPMENT  OF  A  VISUAL  INSPECTION  MODEL 


A .  GENERAL 

1.  Need  for  a  Model  over  Intuition 

Often  the  allocation  of  inspection  time  to  various 

inspection  tasks  is  determined  by  the  inspector.  It  is 

generally  expected  that  an  inspector  will  employ  his  full 

experience,  knowledge,  and  intuition  when  deciding  how  to 

spend  his  time  on  the  various  visual  inspection  tasks  he  must 

perform.  Intuition  is  inadequate  for  such  tasks  as  allocating 

inspection  time.  Hogarth  identifies  the  primary  weakness  in 

reliance  on  intuition  in  general: 

One  could  suppose  that  through  intuitive  judgment  people 
are  able  to  capture  important  information  which  escapes 
codification  and  is  thus  not  amenable  to  statistical 
analysis.  Indeed,  intuitive  judgment  does  capture  much 
information,  however,  it  is  inconsistent.  Moreover  such 
inconsistency  necessarily  attenuates  human  predictive 
validity.  In  other  words,  although  humans  may  on  occasion 
exhibit  high  accuracy  in  judgment,  inability  to  apply 
judgmental  rules  consistently  across  a  series  of  cases 
means  that  on  average  models  outpredict  people.  [Ref  2, 
p. 194] 

Over  the  long  run,  a  model  that  incorporates  all  available 
knowledge  regarding  inspection  phenomenon  and  allocates 
inspection  time  accordingly  will  outperform  any  inspector's 
intuition . 
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A  mathematical  model  of  the  inspection  process  offers 

many  advantages  over  reliance  on  the  intuition  of  an 

inspector.  First,  a  model  offers  a  means  to  understand  the 

underlying  causes  and  effects  occurring  in  the  inspection 

phenomena.  Reliance  on  inspector's  intuition  offers  no 

alternative  method  to  solve  inadequacies  should  intuition 

start  failing.  With  a  model  there  is  the  means  to  test 

alternative  solutions.  Gass  states. 

It  (a  mathematical  model)  enables  us  to  evaluate  the 
effects  of  a  change  of  one  variable  on  all  the  other 
variables;  it  provides  us  with  a  quantitative  basis  to 
sharpen  and  evaluate  our  intuition  of  the  process  under 
investigation. . .  Mathematical  models  enable  us  to  bring 
some  semblance  of  scientific  methodology  to  areas  of 
decision  making  heretofore  characterized  by  intuition  and 
experience.  [Ref  42,  p. 26-27] 

Morse  and  Kimball  write,  "Operations  research  is  a 
scientific  method  of  providing  executive  departments  with  a 
quantitative  basis  for  decisions  regarding  operations  under 
their  control  [Ref  19,  p  7]."  Building  effective  mathematical 
models  of  the  inspection  process  allows  managers  to  take 
control  of  their  inspection  resources  and  make  optimal  use  of 
them  in  support  of  production  facility  goals  and  values.  If 
inspectors  are  not  brought  in  to  support  the  goals  and  values 
of  the  production  facility  they  are  apt  to  pursue  their  own 
set  of  goals  and  values.  Hillier  and  Lieberman  allude  to  what 
then  tends  to  happen  "...(there  is)  a  tendency  for  many 
components  of  an  organization  to  grow  into  relatively 
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autonomous  empires  with  their  own  goals  and  value  systems  [Ref 
5,  p  3] 

Inspector  time  is  a  resource  of  the  manager  and,  as 
such,  it  should  support  the  production  facility's  goals  and 
values.  If  the  manager  can  quantify  his  goals  with  respect 
to  product  quality  then  goal  programming  allows  him  to  model 
his  inspection  time  problem.  The  model  gives  him  the  means 
to  simultaneously  consider  his  goals  and  allocate  inspector’s 
time  to  satisfy  those  goals. 

2.  Modeling  of  the  Inspection  Process 

The  purpose  of  the  inspection  process  is  to  identify 
worker  errors  that  affect  the  system.  The  model  of  the 
inspection  process  must  incorporate  the  effect  that  the  errors 
potentially  have  on  the  system.  Thus,  a  large  part  of 
modeling  the  inspection  process  is  error  analysis. 

The  importance  of  an  error  is  directly  related  to  its 
potential  for  unacceptable  system  effects.  The  Nuclear 
Regulatory  Commission  (NRC)  makes  this  point  in  regard  to 
errors  in  general: 

The  importance  of  an  error  is  a  function  of  its  frequency, 
the  efficiency  of  the  (inspector)  for  the  error,  and  the 
likelihood  and  severity  of  the  potential  consequences  of 
the  error...  [Ref  14,  p  4-17]. 

In  applying  this  same  concept  to  the  realm  of  human  error, 
Pickrel  and  McDonald  write: 
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The  extent  and  cost  of  efforts  to  eliminate  sources  of 
human  errors  should  be  commensurate  with: 

1 )  the  frequency  with  which  the  error  is  expected  to 
occur; 

2)  the  frequency  with  which  a  (system)  failure  will 
occur  as  a  result  of  the  error;  and 

3)  the  possible  consequences  of  the  failure 
condition.  [Ref  27,  p  649] 

Swain  points  the  direction  for  the  modeling  of  the  inspection 
process  when  he  writes: 

Errors  are  important  only  if  they  lead  to  important 
consequences.  Thus,  the  steps  in  error  analysis  are 

a)  Identify  the  consequences  of  interest; 

b)  Identify  all  errors  that  can  lead  to  those 
consequences ; 

c)  Determine  error  rates  for  each  class  of  errors; 

d)  Identify  the  likely  contributing  factors. 

[Ref  10,  p-14] 

The  probability  of  worker  error,  the  probability  of  associated 
inspector  error,  and  the  cost  of  associated  product  flaws  are 
the  parameters  needed  to  establish  the  relative  importance  of 
the  different  errors.  These  parameters  must  then  be  brought 
together  in  one  model  to  represent  the  inspection  process. 

This  chapter  reviews  the  rationale  for  using  the 
opinions  of  subject-matter  experts  to  both  estimate  the 
probability  for  worker  error  and  to  estimate  the  time  required 
to  inspect  for  that  worker  error.  Theories  and  experiments 
dealing  with  the  probability  for  inspector  failure  as  a 
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dealing  with  the  probability  for  inspector  failure  as  a 
function  of  time  to  inspect  and  a  method  for  cost  estimation 
of  system  error  are  reviewed. 

Another  area  discussed  in  the  model  development  is  the 
importance  of  internal  consistency  in  judgments  made  bv 
subject-matter  experts  before  their  subjective  evaluations 
have  merit.  Saaty's  Analytical  Hierarchy  Process  [Ref  17]  is 
reviewed  for  its  method  of  evaluating  the  internal  consistency 
in  a  respondent.  Also,  in  this  chapter,  aoal  programming  is 
reviewed  as  a  means  to  model  problems  such  as  the  allocation 
of  inspection  resources  problem.  The  goal  program  that 
achieves  this  allocation  of  inspector  time  is  relatively 
simple  and  straightforward.  The  challenge  is  developing  the 
probabilities  for  error  and  the  costs  of  error  required  by  the 
model  and  the  goal  program. 

B.  THE  PROBABILITY  OF  CREW  ERROR 

1.  Rationale  for  Subjective  Evaluations 

Research  into  human  reliability  and  error  analysis  led 
to  work  by  the  Nuclear  Regulatory  Commission  (NRC)  in  its 
evaluation  of  human  error  probability  in  the  operation  of 
nuclear  power  plants.  The  NRC  is  at  the  forefront  in  research 
on  human  error  and  design  of  functional,  safe  systems  that 
incorporate  human  error  rates. 
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Traditional  thinking  is  that  personal  opinion  of 
people  with  working  experience  is  worth  little  as  a  basis  for 
good  operations  research  work.  Morse  and  Kimball  were 
emphatic  about  this  when  they  helped  found  operations 
research: 

The  opinions  of  a  few  dozen  persons  who  have  had 
operational  experience  provide  an  extremely  shaky 
foundation  for  any  operations  research. . .The  need  for 
unbiased,  impersonal  facts,  not  opinions,  must  always  be 
borne  in  mind.  [Ref  19,  p  8-9] 

This  influence  is  still  clear  today  as  there  is  often  an 
aversion  to  using  operational  experience  as  a  basis  for 
operations  research.  Echoing  the  sentiments  of  Morse  and 
Kimball  is  the  following  more  recent  comment  by  Von 
Winterfeldt  and  Edwards  that  typifies  the  views  of  many 
systems  analysts  about  the  appropriate  way  to  assess 
probabilities : 

By  far  the  most  common  practical  procedure  for  assessing 
probabilities  is  to  collect  a  large  set  of  observations 
and  then  to  use  a  relative  frequency.  This  procedure  is 
tedious  but  the  existence  of  empirical  science  testifies 
to  its  success. [Ref  29,  p.  110] 

Whereas  probabilities  for  equipment  failure  may  be 
best  established  by  the  strict  empirical  approach,  it  is 
unlikely  that  probabilities  for  human  failure  (human  error) 
can  be  gathered  in  that  manner.  Swain  and  Miller  provide  a 
discussion  of  the  reasons  human  error  data  generally  cannot 
be  obtained  through  empirical  observations  [Ref  1,  p  226]. 
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Horley  and  Giordano  give  a  graphic  example  of  failure  of  the 
empirical  approach  to  obtain  human  error  data  on  artillery  gun 
sections  in  U.S.  Army  tests  conducted  at  Fort  Hood,  Texas  [Ref 
15,  p.67-73].  Swain  and  Guttman  discuss  why  the  empirical 
approach  to  generating  human  error  data  has  not  worked  in 
nuclear  power  plants: 

The  collection  of  relative  frequencies  (for  human  error 
rates  by  empirical  observation)  is  impractical  for  several 
reasons  - 

First,  error  probabilities  for  many  tasks  are  very 
small.  Therefore,  in  any  reasonable  time  period,  not 
enough  errors  will  occur  to  permit  the  fitting  of  a 
meaningful  distribution  to  the  observed  error  relative 
frequencies . 

Second,  usually  there  is  some  penalty  associated  with 
the  commission  of  an  error,  which  discourages  the 
identification  of  individuals  who  commit  the  errors.  If 
such  identification  were  insisted  upon,  there  would  be 
some  unknown,  but  sizeable,  number  of  errors  not  reported. 

Third,  the  administrative  problems  and  costs  of 
recording  and  analyzing  errors  by  person  would  probably 
be  unacceptable.  [Ref  14,  p.  6-2] 

There  are  numerous  activities  in  need  of  human 
reliability  data  for  enhanced  man-machine  system  design.  In 
most  of  these  activities  there  is  extensive  operational 
experience,  but  there  is  little  or  no  human  error  data. 
Expert  judgment  provides  a  means  to  generate  human  error  data. 
Swain  states, 

The  use  of  collective  judgment  by  subject-matter  experts 
is  one  way  to  make  up  for  the  paucity  of  empirical  error 
rate  data.  Studies  indicate  that  it  would  be  feasible  to 
have  a  number  of  people  with  detailed  knowledge  of  various 
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tasks  in  an  industrial  setting  rank-order  tasks  in  terms 
of  error-likeliness .  [Ref  10,  p.101] 

Miller  and  Swain  briefly  describe  how  the  methods  that  use 

expert  judgment  to  generate  human  error  data  are  structured: 

Subject-matter  experts  judge  the  relative  or  absolute 
likelihood  of  error  for  several  task  descriptions,  and 
these  responses  are  converted  to  usable  human  error 
probabilities  by  mathematical  scaling  techniques.  [Ref  1, 
p.  238] 

The  NRC  endorses  four  methods  for  eliciting  expert 

judgment  input  to  estimate  human  error  probabilities.  The  NRC 

makes  the  following  exhortation  in  regards  to  these  methods: 

Make  the  best  use  possible  of  current  procedures  and 
information,  even  though  they  may  be  flawed. . .These  (the 
four  methods  endorsed  by  the  NRC)  are  based  on  sound 
psychological  theory  and  empirical  support  that  indicate 
their  potential  usefulness.  And  as  a  practical  matter, 
they  can  be  used  now  to  obtain  needed  human  error 
probabilities.  [Ref  14,  p  8-6] 

The  NRC  approved  methods  are  the  paired  comparisons  procedure, 
ranking  and  rating  procedure,  direct  numerical  estimation 
procedure,  and  indirect  numerical  estimation  procedure.  Swain 
and  Guttman  [Ref  14,  p.8-6  to  8-13]  and  Seaver  and  Stillwell 
[Ref  28,  p  A-2  to  A-51]  provide  detailed  explanations  of  each 
of  the  NRC  approved  methods. 

2.  Experts  to  Supply  Expert  Judgment 

Embrey  noted  that  his  whole  approach  to  developing 
human  error  probabilities  by  subjective  input  was  contingent 
on  judges.  He  writes: 

The  success  of  the  approach  is  likely  to  be  critically 
dependent  on  the  knowledge  and  experience  that  the  judges 
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possess  regarding  the  tasks  they  are  asked  to  assess.  For 
this  reason  it  is  very  important  that  strenuous  attempts 
are  made  to  obtain  appropriately  qualified  judges.  [Ref 
28,  p.  30] 

In  commenting  about  eliciting  judgments  of  uncertainty  (or 
probabilities)  from  people.  Von  Winterfeldt  and  Edwards  write: 


If  a  numerical  measure  of  uncertainty  is  an  opinion,  it 
characterizes  a  person  as  well  as  an  event;  we  want  to 
elicit  from  the  right  person.  Who  is  that?.. [Ref  29,  p. 
113] 

The  "right  person"  is  the  one  who  has  the  best  understanding 
of  the  phenomenon  under  investigation  and  who  can  best  relate 
that  understanding  in  terms  that  can  be  quantified. 

Swain  offers  that  the  worker  or  inspector  himself  is 
an  expert  on  error  rates: 

Subject-matter  experts  are  persons  who  are  especially 
knowledgeable  about  the  tasks  for  which  error  rates  are 
to  be  derived.  Thus,  if  one  wanted  to  derive  error  rate 
estimates  for  new  tasks  in  an  assembly  plant,  the  experts 
would  be  persons  who  perform  or  had  performed  similar 
tasks  in  an  assembly  plant.  [Ref  10,  p.101] 

Saaty  agrees.  Saaty  not  only  allows  that  the  worker  or 

inspector  is  an  expert  in  the  operational  aspects  of  his  work, 

but  he  implies  that  he  also  is  the  best  person  for  relating 

to  others  phenomena  about  that  work: 

We  generally  agree  that  people  who  experience  a  phenomenon 
firsthand  are  the  ones  who  can  best  shed  light  on  our 
understanding  of  it;  indeed,  knowledge  derived  from 
experience  is  basic  to  all  understanding.  [Ref  17,  p.  9] 
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Lai,  Shenoi,  and  Fan  suggest  that  workers  and  inspectors  are 

specifically  well-equipped  to  estimate  probabilities 

concerning  errors  and  hazards: 

One  of  the  major  problems  facing  many  quantitative  risk 
assessment  studies  is  the  lack  of  proper  data  about  the 
probabilities  of  occurrence  of  hazardous  events.  Often, 
the  only  alternative  is  to  employ  rough  estimates  provided 
by  operations  personnel....  Human  beings  often  possess 
quality  knowledge  that  arises  out  of  their  insights  and 
experiences.  When  the  safety  of  a  system  is  being 
examined  this  expertise  has  the  greatest  weight.  [Ref  40, 
p . 136] 

While  the  worker  and  inspector  are  certainly  experts 
on  the  operational  aspect  of  the  human  error  phenomenon  that 
is  a  subject  of  this  study,  the  question  remains  whether  they 
can  relate  this  expert  understanding  in  terms  that  permit 
quantification. 

3.  Indirect  Numerical  Estimation  Technique 

Miller  and  Swain  write  that  a  major  disadvantage  of 
methods  that  use  expert  judgment  to  generate  human  error  data 
is  that  the  expert  often  lacks  sufficient  familiarity  with 
probabilities  to  be  able  to  give  meaningful  input  [Ref  1,  p. 
244].  Persons  with  operational  experience,  such  as  workers 
and  inspectors,  are  often  not  qualified  to  give  a  direct 
numerical  estimate  of  a  probability.  Paulos  states,  "An 
appreciation  for  probability  takes  a  long  time  to  develop. 
[Ref  7,  p.133]."  Morse  and  Kimball  warn,  "People  seldom 

estimate  random  events  correctly ...  their  opinions  are  nearly 
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always  unconsciously  biased  [Ref  19,  p.  8]."  Swain  cautions: 

It  is  not  suggested  that  (subject-matter)  experts  be  asked 
to  make  estimates  of  actual  error  rates  for  tasks.  Such 
estimates  would  tend  to  vary  widely  and  would  generally 
underestimate  the  true  error  rates.  Some  subject-matter 
experts  simply  would  refuse  to  make  such  estimates.  [Ref 

10,  p  101] 

Hogarth  states. 

People  do  have  difficulty  in  expressing  their  degree  of 
knowledge  in  the  precise  quantitative  form  demanded  by 
probability  theory. . . .  Numerous  studies  have  shown 

fallibilities  in  intuitive  attempts  to  guess  probabilistic 
relations  from  given  facts.  [Ref  2,  p.192] 

Embrey  shows  that,  when  giving  uncertainty  judgments, 

respondents  do  significantly  better  when  dealing  with 

likelihood  ratios  and  odds  rather  than  probabilities.  Embrey 

concludes,  "that  judges  (subject-matter  experts)  are  best  at 

making  simple  directional  judgments  of  the  form  'task  A  is 

more  likely  to  be  successful  than  task  B'  [Ref  28,  p  9]." 

Swain's  work  supports  Embrey ' s .  Swain  adds: 

Persons  who  will  not  guess  at  the  probability  of  a 
particular  event  will  readily  rank-order  several  events 
in  terms  of  increasing  difficulty,  hazard,  or  other 
dimension,  and  they  will  do  this  ranking  with  great 
confidence.  [Ref  10,  p.101] 

Killer  and  Swain  comment,  "Humans  are  very  capable  of  making 
qualitative  judgments  along  one  dimension  [Ref  1,  p.  238]." 

Estimation  methods  for  human  error  probability  that 
allow  a  person  without  probability-estimating  skills  to  relate 
his  expert  understanding  of  error  phenomena  by  simple 
directional  judgments  could  be  very  useful.  Of  the  four 
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expert  judgment  methods  proposed  by  the  NRC,  two  methods  are 
based  upon  simple  directional  judgments:  paired  comparisons 
and  indirect  numerical  estimation. 

A  problem  with  paired  comparisons  is  the  large  number 
of  judges  required  to  obtain  valid  estimates  of  the  task 
positions  on  a  subjective  scale.  The  relative  frequency  by 
which  one  task  is  rated  more  likely  to  fail  than  another  by 
a  large  pool  of  experts  is  what  leads  to  a  scaled  value  for 
that  task.  If  there  are  only  five  judges  providing  input  then 
there  are  insufficient  observations  to  be  significant. 

The  indirect  numerical  estimation  method  of  obtaining 
human  error  probabilities  from  expert  judgement  is  a  robust 
procedure  and  is  the  primary  procedure  for  estimating  human 
error  rates  pursued  in  this  thesis.  Like  paired  comparisons, 
it  depends  mostly  on  human  qualitative  versus  quantitative 
judgments.  The  indirect  numerical  estimation  method  asks  the 
respondent  two  questions  concerning  task  failure  in  his  area 
of  expertise: 

1 )  Is  this  task  error  more  or  less  likely  than  that  task 

error? 

2 )  How  much  more  or  how  much  less  likely  is  the  first 
task  error  versus  the  second  task  error? 

A  detailed  discussion  of  the  conversion  of  these  expert 

judgments  to  human  error  data  by  the  indirect  numerical 
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estimation  method  is  given  by  Seaver  and  Stillwell  [Ref  12] . 
A  summary  of  that  process  is  given  in  the  example  below. 

Using  the  one-by-one  comparisons  given  by  a 
respondent,  a  subjective  scale  of  likelihood  is  generated. 
Assume  a  respondent  gives  these  likelihoods  for  four  mutually 
exclusive,  independent  task  errors  that  encompass  the  entire 
set  of  errors  that  can  occur  on  a  given  system: 

Task  A  error  is  twice  as  likely  as  Task  B  error. 

Task  B  error  is  just  as  likely  as  Task  C  error. 

Task  C  error  is  three  times  as  likely  as  Task  D  error. 

The  likelihood  of  task  error  for  all  tasks  can  simply  be 
written  in  terms  of  any  one  task  error.  In  the  example,  use 
D  as  the  base  scale: 

Task  A  error  =  6  X  (Task  D  error) 

Task  B  error  =  3  X  (Task  D  error) 

Task  C  error  =  3  X  (Task  D  error) 

Task  D  error  =  1  X  (Task  D  error) 

If  the  probability  of  any  one  task  error  is  known  or  estimated 
from  some  other  source  (Task  D  fcr  example)  then  the 
probabilities  of  all  four  events  are  known. 

If  the  error  probabilities  are  normalized,  then  the 
probability  for  error  in  any  one  task  is  known,  given  that  an 
error  occurs.  Normalization  takes  place  in  the  example 
problem : 
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6D  +  3D  +  3D  +  ID  =  1.00 


D  =  .0769 

Therefore,  given  an  error  occurs  on  the  system,  the 
probability  that  it  was: 

Task  A  error  is  46% 

Task  B  error  is  23% 

Task  C  error  is  23% 

Task  D  error  is  8% 

These  probabilities  for  task  error  apply  to  one  respondent. 

The  next  topic  concerns  the  number  of  respondents 

needed  to  generate  good  point  estimates  of  the  error  data 

being  studied.  Von  Winterfeldt  and  Edwards  state,  "Both 

theory  and  evidence  teach  us  to  use  more  than  one  respondent 

whenever  we  can.  [Ref  29,  p.113)."  Guttman  and  Swain, 

commenting  on  the  indirect  numerical  estimation  technique, 

cite  the  work  of  Von  Holstein  in  stating,  "as  with  direct 

numerical  estimation,  little  is  gained  by  using  more  than  six 

experts  [Ref  14,  p  A-46]."  Hogarth  writes: 

...judgments  between  individuals  are  likely  to  be 
correlated.  Thus  one  is  often  averaging  redundant 
information  with  the  result  that  beyond  a  certain  point 
gains  in  adding  experts  to  a  group  are  small.  It  has  been 
estimated  that  in  most  predictive  situations  involving  the 
use  of  experts  there  is  little  point  in  averaging  more 
than  about  ten  persons  and  that  six  or  seven  will  often 
be  sufficient.  [Ref  2,  p  195] 

The  next  subject  is  the  aggregating  of  several 
individual  likelihood  estimates  to  a  single  estimate 
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applicable  to  the  entire  production  facility.  Many  schemes 

exist  for  combining  several  likelihood  estimates  into  a  single 

estimate.  Von  Winterfeldt  and  Edwards  offers  this  insight: 

Since  proper  scoring  rules  are  convex  functions  on  the 
probability  simplex,  the  score  of  the  average  of 
individual  probabilities  will  necessarily  be  better  than 
the  average  of  the  individual's  scores.  [Ref  29,  p.133] 

The  authors  continue  with  this  straightforward  recommendation: 

Simply  elicit  the  desired  uncertainty  measures  from  each 
member  individually....  Transform  the  results  into 
probabilities  and  average  them.  The  odds  seem  excellent 
that,  if  you  do  anything  more  complex,  you  will  simply  be 
wasting  your  effort.  [Ref  29,  p.136] 

This  thesis  pursues  the  method  recommended  by  Von  Winterfeldt 

and  Edwards . 

4.  Consistency  in  Expert  Judgment 

Workers  and  inspectors  are  typically  trained  to  simply 
perform  the  tasks  required  in  their  jobs.  They  are  not 


routinely 

trained 

to 

make  quantitative 

judgments 

about 

phenomena 

relating 

to 

their  work.  When 

asked  to 

make 

quantitative  judgments  it  can  be  expected  that  some  personnel 
are  overwhelmed  with  the  requirement  and  thus  fail  to  provide 
valid  input.  It  would  be  helpful  to  have  a  means  to  screen 
the  incoming  data  to  eliminate  surveys  of  those  respondents 
who  were  incapable  or  unwilling  to  provide  valid  data. 

Consistency  on  the  part  of  a  respondent  is  some 
measure  of  objectivity,  internal  logic,  and  operational 
experience.  Saaty  defines  the  term: 
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The  meaning  of  consistency  is  that  the  intensities  of 
relations  among  ideas  or  objects  based  on  a  particular 
criterion  justify  each  other  in  some  logical  way.  (Ref 
17,  p  18) 

Some  measure  of  consistency  is  necessary  to  establish  the 

credibility  of  the  expert  whose  judgments  will  eventually 

affect  a  final  decision.  Saaty  states,  "We  do  not  want  the 

decision  to  be  based  on  judgments  that  have  such  low 

consistency  that  they  appear  random  (Ref  17,  p.82).M 

Stillwell  and  Seaver  add,  "High  levels  of  inconsistency  would 

suggest  the  entire  judgmental  process  is  suspect  (Ref  13, 

p.32)."  A  way  to  identify  inconsistent  respondents  is  to 

require  them  to  make  more  pairwise  ratio  judgments  than 

absolutely  necessary  and  then  use  those  judgments  to  check  the 

respondent's  internal  consistency.  Seaver  and  Stillwell 

describe  how  this  works  for  the  indirect  numerical  estimation 

method  of  obtaining  human  error  probabilities: 

To  obtain  a  complete  set  of  probabilities  for  N  events 
only  requires  a  minimum  of  N-l  such  judgments  and  an 
independent  estimate  of  the  human  error  probability  of 
one  event.  Additional  judgments,  however,  may  be  obtained 
as  consistency  checks.  [Ref  12,  p  A-46] 

When  more  pairwise  ratio  judgments  are  made  by  a  respondent 

than  is  absolutely  necessary  there  will  most  likely  be  some 

inconsistency.  Stillwell  and  Seaver  remind  that,  "Some 

inconsistencies  enter  into  all  judgmental  processes.  (Ref  13, 

p.32)"  Saaty  argues  that  some  amount  of  inconsistency  is 
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necessary  as  humans  adapt  to  an  ever-changing  world.  Saaty 
states : 

We  may  not  be  perfectly  consistent,  but  that  is  the  way 
we  tend  to  work.  It  is  also  the  way  we  grow.  When  we 
integrate  new  experiences  into  our  consciousness,  previous 
relationships  may  change  and  some  consistency  is  lost. 
[Ref  17,  p  82] 

A  level  of  consistency  is  desirable  to  establish 
respondent  validity  and  expertise.  A  level  of  inconsistency, 
however,  is  to  be  expected  among  human  judgment.  Saaty 
comments,  "As  long  as  there  is  enough  consistency  to  maintain 
coherence  among  the  objects  of  our  experience,  the  consistency 
need  not  be  perfect  [Ref  17,  p.82]."  There  is  a  definite 
need  to  establish  an  acceptable  level  of  inconsistency  in  a 
judge,  beyond  which  the  judgments  are  discarded  as  invalid. 

To  obtain  some  measure  of  consistency  Saaty  suggests 
the  development  of  an  (N  X  N)  reciprocal  matrix  where  N 
represents  the  number  of  items  for  comparison.  A  respondent 
makes  a  ratio  comparison  between  every  two  possible  items  such 
that  a  total  of  (N)  X  (N-l)/2  judgments  are  made.  The  (N  X 
N)  matrix  is  filled  in  with  ones  down  the  main  diagonal,  the 
(N)  X  (N-l)/2  judgments  fill  the  lower  triangular  matrix  below 
the  diagonal  of  ones,  and  the  reciprocals  of  the  judgments 
fill  the  upper  triangular  matrix  above  the  diagonal  of  ones. 

The  maximum  eigenvalue  (XBax)  for  this  matrix  is  a 
measure  of  consistency  of  the  judgments  in  the  matrix.  Saaty 
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proves  that,  "A  positive  reciprocal  matrix  is  consistent  if 
and  only  if  Amax  is  equal  to  N.  (Ref  16,  p.181)"  Saaty  then 
turns  Amax  into  a  consistency  index  (Cl)  by  the  following 
operation: 

(2-1)  [  Amax~N  ]  /  (  N-l  ]  =  Cl. 

Saaty  suggests  that  comparing  the  Cl  of  a  set  of  respondents' 
judgments  against  that  Cl  found  by  a  randomly  generated 
reciprocal  matrix  should  give  a  general  measure  of  the 
consistency  called  the  consistency  ratio  ( CR ) : 

(2~2.)  (  CIrespon(jent  )  /  ( CIrandoB1  B8trix )  =  CR . 

The  random  entries  in  the  random  reciprocal  matrix  are  chosen 
by  computer  from  the  range  of  possible  values  used  by  the 
respondents  on  their  original  survey.  Several  hundred  random 
matrices  are  computed  with  respective  CIs  for  each.  Those  CIs 
are  then  averaged  to  obtain  a  mean  Cl  for  a  random  matrix. 
Appendix  D  shows  the  table  for  judgments  limited  to  a  one  to 
nine  scale  and  describes  the  generation  of  the  table  in  more 
detail . 

Saaty  provides  a  general  rule  for  acceptable 
consistency  ratio  values  with  which  he  had  success  in  his 
work.  Saaty  maintains  that  a  consistency  ratio  less  that  0.1 
demonstrates  acceptable  consistency  and  that  a  respondent 
providing  such  consistency  is  giving  judgments  worth 
considering  in  decision-making.  [Ref  17,  p.  83] 
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A  disadvantage  of  Saaty's  method  is  the  number  of 
judgments  required  of  each  respondent.  In  a  4  X  4  matrix  an 
expert  makes  only  six  judgments  [(4X3)  /  2] .  In  a  10  X  10 
matrix  an  expert  must  make  45  judgments  [(10  X  9)  /  2].  Saaty 
suggests  extensive  use  of  hierarchies  so  that  the  number  of 
judgments  are  reduced.  For  example,  if  the  10  X  10  matrix 
could  be  partitioned  to  sets  of  four,  three,  and  three  for 
pairwise  ratio  comparisons  then  the  number  of  judgments  is 
reduced  from  45  to  15.  Figure  1  depicts  the  reduction  of 
required  judgments  that  is  accomplished  by  use  of  a  hierarchy. 

Saaty's  method  is  the  method  pursued  in  this  thesis 
fcr  establishing  sufficiently  consistent  input  for 
consideration  in  decision  making. 

C.  THE  PROBABILITY  OF  INSPECTOR  ERROR 
1.  Inspectors  Make  Mistakes 

Traditionally  inspectors  are  viewed  as  a  panacea  for 
production  problems.  Many  production  and  quality  assurance 
plans  are  premised  on  an  assumption  of  100%  accuracy  on  the 
part  of  inspectors.  Inspectors  are  not  perfect.  Swain  makes 
this  clear. 

There  is  no  point  in  arguing  whether  people  should  make 
errors.  We  all  do.  Human  error  rates,  like  material 
failure  rates,  are  merely  facts  [Ref  10,  p.5]. 


Total  Comparisons  Required  -  45 


Total  Comparisons  Required 

6*a-*o*a-i5 


N  =  (n)  x  (n-1)/2 


whorj  N  *  numbn  or  comparisons  needed 
to  establish  Interval  aoale 
using  the  Pairwise  Comparisons  Method. 

n  ■  number  of  Items  to  be 
put  In  Interval  soale. 


Figure  1  -  Reduction  of  Pairwise  Judgments 
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Drury  makes  this  call  to  action: 


Intrinsically,  man  is  rarely  perfect  in  either  detection 
or  diagnosis.  .  .This  has  led  in  recent  years  to  studies 
which  have  demonstrated  the  effects  inspector’s  errors  can 
have.  .  .and  the  need  to  take  account  of  realistic  values 
of  inspector's  errors.  [Ref  9,  p.ll] 

To  decrease  the  error  rate  some  form  of  data  collection  is 

necessary  to  make  a  quantitative  assessment  of  inspector 

error.  Bennett  emphasizes  this  point: 

So  what  must  be  done  to  achieve  quality  objectives  given 
that  inspection  error  is  to  be  ever  present?  The  answer 
is  simple.  The  quality  control  engineer  must  be  able  to 
accurately  measure  these  errors  and  then  be  able  to  design 
for  them.  [Ref  9,  p.l] 

2.  The  Direct  Relationship:  the  More  Time  Inspector 
Takes,  the  Greater  the  Inspector  Accuracy 

An  appealing  method  for  viewing  inspector  error 

probability  was  introduced  by  Wreathall  in  work  for  the  NRC. 

Wreathall's  Operator-Action  Tree  method  (OAT)  is  depicted  in 

Figure  2. 

The  OAT  approach  is  based  on  the  assumption  that  human 
response  to  an  environmental  event  consists  of  three 
activities : 

1 )  perception, 

2)  diagnosis,  and 

3)  response. 

The  basic  operator-action  tree  is  based  on  the  potential 
for  error  in  each  of  the  three  activities. 

The  second  major  assumption  is  that  time  available  for 
diagnosis  is  the  dominant  factor  determining  the 
probability  for  failure.  That  is,  given  a  short  period 
of  time,  people  will  fail  to  diagnose  a  situation 
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correctly  more  often  than  when  given  a  longer  period.  [Ref  1, 
p.236] 


BASIC  OPERATOR  ACTION  TREE 

OPERATOR  OPERATOR  OPERATOR 

EVENT  OBSERVES  DIAGNOSES  CARRIES  OUT  SUCCESS/ 

OCCURS  INDICATIONS  PROBLEM  REQUIRED  RESPONSE  FAILURE 

-  a 

-  .  -  p 

- F 

_______ -  F 

SOURCE:  NURE0/CR-S010 

Figure  2  -  Operator-Action  Tree 
The  intuitive  appeal  for  the  first  assumption  is  overwhelming. 
An  inspector  must  "see",  or  visually  fixate,  on  the  event. 
He  must  be  trained  sufficiently  to  be  able  to  discriminate 
error  in  the  object  of  his  fixation.  Finally,  the  inspector 
must  be  able  to  respond  properly  to  overcome  the  error. 
Failure  to  accomplish  any  one  of  these  three  steps  will  result 
in  inspector  failure  on  the  task. 

Not  only  does  the  second  assumption,  that  less  time 
available  means  more  inspector  errors,  make  intuitive  sense, 
but  there  is  strong  experimental  basis  for  it.  Drury's  work 
with  flat  glass  inspectors  [Ref  26,  p.265].  Buck's  experiments 
with  Landolt  rings  [Ref  9,  p.169],  and  Rizzi's  tests  with 

flawed  dot  patterns  on  cards  [Ref  25,  p.279]  convincingly 

demonstrate  that  inspector  errors  increase  with  less  exposure 
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time  to  the  object  being  inspected.  Drury  summed  up  his  work 

and  that  of  Smith  and  Barany  [Ref  22]  when  he  wrote: 

In  all  of  the  studies,  a  similar  effect  is  observed,  in 
that  as  more  time  is  allowed  to  inspect  each  item,  the 
probability  of  rejecting  a  faulty  item  increases  whilst 
the  probability  of  accepting  a  good  item  decreases.  [Ref 
32,  p.  6] 

Drury  suggests  something  akin  to  OAT: 

The  inspection  mechanism  postulated  is  a  visual  search 
until  either  a  potential  defect  is  found  or  time  runs  out 
for  that  item,  followed  by  a  decision  about  the  potential 
defect  as  to  whether  it  is  acceptable  or  rejectable.  [Ref 
32,  p.  6] 

Just  as  the  OAT  method  suggests,  a  visual  search  must  take 
place  until  the  inspector  "sees"  the  prospective  error.  This 
success  of  the  visual  search  is  no  doubt  a  function  of  many 
factors.  The  experimental  evidence  reviewed  points  strongly 
to  time  available  to  the  inspector  as  the  most  significant 
factor  to  whether  the  search  v.’ill  be  succo^f jl  or  not.  If 
the  search  leads  to  a  prospective  error  then  the  inspector 
must  decide  whether  to  accept  or  reject  the  object  bearing  the 
prospective  error.  This  decision  involves  experience  of  the 
inspector,  remaining  time  available  to  the  inspector,  and 
costs  and  payoffs  relating  to  finding  errors. 

Smith  and  Barany  found  signal  detection  theory  helpful 
in  describing  the  decision  an  inspector  must  make  [Ref  22, 
p.300].  Drury  also  used  signal  detection  theory  to  model 
inspector  decision  performance  [Ref  32,  p.2-6].  Chapman  and 
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Sinclair  give  this  opinion  of  the  application  of  signal 

detection  theory  to  their  study  of  inspection  work: 

Perhaps  the  most  important  thing  to  emerge  from  this  study 
is  the  usefulness  of  the  theory  of  signal  detection  in 
industrial  inspection  situations  both  from  a  theoretical 
and  a  practical  point  of  view.  The  theoretical  value  lies 
in  the  insights  it  provides  into  an  inspection  task,  and 
the  practical  value  arises  from  the  fact  that  it  allows 
economic  justification  for  the  application  of  ergonomics 
to  inspection,  and  the  rel-ative  ease  with  which 
recommendations  for  improvement  can  be  derived.  [Ref  9, 
p.  241] 

The  basics  of  signal  detection  theory  were  established 
by  the  pioneering  work  of  Tanner,  Swets,  Birdsall,  and  Green 
between  1954  and  1966  [Ref  8,  p.18].  Figure  3  is  the  common 
depiction  of  the  theory  applied  to  the  inspector's  decision 
[Ref  9,  p.14].  The  detection  of  signals  is  assumed  by  the 
theory  to  involve  two  processes:  discrimination  (d')  and 
decision  (/?).  Discrimination  (d')  requires  the  inspector  to 
be  capable  of  distinguishing  between  success  and  error  on  a 
task  he  is  inspecting. 

Discrimination  is  affected  by  such  things  as  the 
experience  of  the  inspector  (more  experience  tends  to  make  d' 
larger)  and  the  time  available  to  perform  the  inspection  (more 
time  tends  to  make  d'  larger).  Decision  (/?)  requires  the 
inspector  to  accept  or  reject  what  he  sees.  A  decision  (where 
the  point  is  between  the  two  modal  values  of  the  Gaussian 
curves)  is  influenced  by  such  things  as  the  inspector's 
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Figure  3  -  Signal  Detection  Theory  [Ref  9,  p.14] 
subjective  probability  that  an  error  will  occur  (if  he  thinks 
errors  are  very  likely  then  he  is  more  likely  to  reject)  and 
his  attitude  toward  risk.  [Ref  8,  p.18-21] 

Smith  and  Barany  explain  in  terms  of  signal  detection 
theory  what  transpires  to  cause  a  reduction  in  inspector 
accuracy  with  a  decrease  in  time  available.  Smith  and  Barany 
write : 

If  the  pace  of  the  inspection  task  is  increased,  then  the 
inspector  will  have  less  time  in  which  to  make  his 
observation,  and  his  observing  mode  will  tend  toward  that 
described  as  "blurred  observing" . [Ref  22,  p.300] 

The  effect  of  "blurred  observing"  is  to  decrease 

discriminability  (d’).  With  discriminability  (d')  reduced 

there  is  more”  overlap  of  the  two  Gaussian  curves  (see  Figure 

3)  and  whatever  decision  (p)  is  made  must  result  in  either 
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more  Type  I  or  Type  II  errors.  Smith  and  Barany  conducted 
experiments  which  demonstrated  what  signal  detection  theory 
postulated. 

The  experimental  results  from  Drury  [Ref  26];  Rizzi, 
Buck,  and  Anderson  [Ref  25];  and  Smith  and  Barany  [Ref  22] 
noted  above  are  for  relatively  simple  inspection  tasks  along 
one  dimension  (flaws  in  sheet  glass,  numbers  of  holes  in 
disks,  dots  on  cards)  as  opposed  to  more  complex  tasks  with 
a  multitude  of  possible  errors  on  several  dimensions  (circuit 
boards,  silicon  chips).  For  simple  inspection  tasks  the 
experimental  results  clearly  show  a  direct  relationship 
between  inspector  time  to  observe  and  inspector  accuracy. 
Schoonard  emphasized  this  point  with  an  experiment  where  seven 
different  classes  of  errors  were  possible  on  circuit  boards. 
The  inspection  tasks  for  the  seven  classes  were  ranked  from 
simplest  to  most  difficult.  A  deliberate  slowing  of  the 
inspection  time  resulted  in  marked  improvement  on  the  more 
simple  inspection  tasks  but  not  so  great  an  improvement  on  the 
other,  more  difficult  inspection  tasks.  [Ref  24,  p.  368-376] 
Drury  predicted  this  finding  by  signal  detection  theory 
principles.  In  reference  to  simple  inspection  tasks  Drury 
wrote , 

For  simple  discriminations  (such  as  the  Smith  and  Barany 
experiments)  the  discriminability  will  be  high  er ough  to 
give  almost  perfect  inspection  as  the  time  allowed  is 
increased.  [Ref  32,  p.7] 
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It  is  clear  from  experimental  results  that  simple  inspection 
task  accuracy  improves  with  time. 

The  nature  of  the  relationship  between  time  taken  by 
inspector  and  his  accuracy  was  explored  by  both  Rizzi,  Buck, 
and  Anderson  [Ref  25,  p.  279]  and  Drury  [Ref  26,  p.265]. 
Drury  found  an  exponential  relationship  between  the 
cumulative  probability  of  locating  a  flaw  in  sheet  glass  and 
time  taken  to  inspect.  Rizzi,  Buck,  and  Anderson  conducted 
tests  on  subjects  inspecting  small  cards  with  dots.  Similar 
to  Drury's  results,  Rizzi,  Buck,  and  Anderson  also  found  the 
inspector's  exposure  time  to  the  object  exerted  an  exponential 
effect  on  the  probability  of  a  correct  detection.  Figure  4 
shows  the  results  of  Drury's  and  Smith  and  Barany’s  work  with 
the  effect  of  time  on  the  probability  of  inspector  accuracy. 
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Drury  [Ref  26,  p.264]  Smith  and  Barany  [Ref  32,  p.5] 

Figure  4  -  Inspector  Accuracy  as  a  Function  of  Time 
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3.  Limits  to  the  Direct  Relationship 

It  would  seem  that  there  must  be  some  limit  to  the 
direct  relationship  that  more  time  spent  by  an  inspector 
yields  more  inspector  accuracy.  This  section  reviews 
experimental  results  that  confirm  both  lower  and  upper  bounds 
to  this  direct  relationship. 

a.  Minimum  Time  is  Limited  by  Time  Required  for 
Visual  Fixation 

The  lower  bound  for  this  direct  relationship  is 
the  physical  limitation  of  the  human  eye  to  focus  and  fixate 
on  any  visual  stimuli  and  the  brain  to  comprehend  what  the  eye 
is  seeing.  Experimental  results  revealing  what  might  be  a 
minimum  time  for  the  eye  to  fixate  and  the  brain  to  begin  to 
function  for  inspection  purposes  are  offered  by  Drury.  Drury 
concluded  from  his  experimental  results  that  it  took  seven 
tenths  of  a  second  (700  milliseconds)  for  his  subjects  to  open 
and  focus  their  eyes  and  begin  searching  for  errors.  [Ref  26, 
p.  260].  During  that  initial  time  from  zero  to  seven  tenths 
of  a  second  there  is  no  increase  in  inspector  accuracy. 

b.  Maximum  Time  is  Limited  by  Human  Vigilance 
The  upper  bound  for  the  direct  relationship  of 

more  time  resulting  in  more  accuracy  is  the  result  of 
decreased  vigilance  that  occurs  when  an  inspector  exceeds  his 
attention  span.  Chapman  and  Sinclair  [Ref  9],  Kochhar  and 
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Jaisingh  [Ref  23],  and  Schoonard,  Gould,  and  Miller  [Ref  24] 
demonstrated  upper  bounds  to  the  direct  relationship  that  was 
found  by  Drury  [Ref  26];  Rizzi,  Buck,  and  Anderson  [Ref  25]; 
and  Smith  and  Barany  [Ref  22] .  Experiments  by  Chapman  and 
Sinclair  of  experienced  personnel  inspecting  poultry  carcasses 
at  a  meat  processing  plant  [Ref  9,  p.231]  and  the  work  of 
Kochhar  and  Jaisingh  with  experienced  inspectors  checking 
products  on  a  CRT  screen  [Ref  23,  p.  44]  are  two  examples  of 
note.  Both  showed  initial  improvement  in  inspector  accuracy 
as  time  was  increased,  just  as  Drury  and  others  had  noted. 
At  a  point,  however.  Chapman  and  Sinclair  [Ref  9]  and  Kochhar 
and  Jaisingh  [Ref  23]  reported  a  degradation  in  inspector 
accuracy  when  inspectors  were  made  to  take  more  time  in  their 
visual  inspection  tasks.  Schoonard,  Gould,  and  Miller 
reported  similar  findings  with  their  experienced  inspectors 
who  were  checking  circuit  boards  for  any  of  seven  different 
classes  of  errors  that  might  be  present.  As  Schoonard,  Gould, 
and  Miller  increased  the  inspection  time  to  1.5  times  the 
standard  time  the  inspectors  spent  on  their  tasks,  significant 
improvement  resulted  across  all  seven  classes  of  error-types. 
With  any  additional  increases  over  the  1.5  times  the  standard 
time  for  the  inspectors,  there  was  no  improvement  in  inspector 
accuracy.  [Ref  24,  p.365].  Chapman  and  Sinclair  offer  a 
possible  explanation  for  this  degradation  phenomenon  with 
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increase  in  inspector  time: 

. . .during  the  course  of  such  a  task  periods  of  inattention 
occur  which  become  more  frequent  or  prolonged  as 
presentation  time  increases....  A  possible  answer  (for  the 
prolonged  periods  of  inattention)  may  be  that  the  human 
organism  maintains  a  homeostatic  level  of  arousal,  and  if 
this  is  dependent  on  incoming  stimuli,  it  could  be  argued 
that  at  the  lower  speeds  additional  stimuli  are  required 
which  are  obtained  during  the  extended  periods  of 
inattention  to  the  task.  [Ref  9,  p  249] 

Kochhar  and  Jaisingh  offer  this  explanation  of  the  decline  in 

inspector  accuracy  resulting  from  increased  inspection  time 

available  that  was  noted  in  their  experiments: 

Error  detection  in  vigilance  situations  of  this  type  is 
dependent  upon  operator  arousal  levels.  Thus  the  low 
levels  of  fault  information  and  product  pacing  did  not 
arouse  the  subject  to  the  point  where  performance  could 
be  optimal....  It  could  be  inferred  that  operator 
performance  in  inspection  is  related  to  arousal  and  in 
evaluating  trade-offs  between  the  different  variables  due 
consideration  should  be  given  to  maintaining  operator 
arousal.  [Ref  23,  p.  44] 

The  phenomenon  that  more  time  beyond  a  certain  critical  point 
does  not  yield  more  inspector  accuracy  is  real.  Analysts 
cannot  blindly  assume  that  more  inspector  time  will  continue 
to  yield  an  improving  probability  for  accuracy.  To  continue 
to  allot  more  inspector  time  is  counterproductive.  In 
addition  to  an  actual  decrease  in  inspector  accuracy,  more 
inspector  time  costs  the  facility  either  the  salary  of  more 
inspectors  or  reduced  productivity  due  to  a  slowed  production 
line . 
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4.  Significance  of  Schoonard's  Findings 

Many  visual  inspection  experiments  have  been 

conducted.  Few  of  those  experimental  findings  have  direct 

application  to  the  actual  inspection  operations  on  production 

lines.  Schoonard,  Gould,  and  Miller  explain  the  shortcomings 

of  the  bulk  of  visual  inspection  experiments: 

Fundamental  laboratory  studies. .  .  have  limited  application 
to  most  actual  inspection  situations  because  they  use 
discrete  targets  on  homogenous  backgrounds,  whereas  most 
inspection  situations  consist  of  poorly  defined  targets 
on  non-homogenous  backgrounds.  [Ref  24,  p  365] 

Another  shortcoming  of  much  experimental  work  is  the 

hiring  of  students  to  perform  the  simple  visual  inspection 

tasks  in  the  laboratory  studies.  The  nature  of  most  actual 

inspection  tasks  is  sufficiently  technical  to  require 

specially  trained  inspectors.  The  inspectors'  frames  of  mind, 

motivation,  and  dedication  have  much  to  do  with  how  they  will 

perform  when  inspecting.  The  results  will  very  likely  differ 

significantly  between  experiments  where  fully  paid  and  trained 

inspectors  perform  technical  inspection  tasks  that  they 

perform  everyday  and  other  experiments  where  students  are  paid 

some  nominal  fee  to  perform  very  simplistic  visual  inspection 

tasks  for  a  day.  To  extrapolate  information  from  experimental 

work  for  use  in  analysis  of  actual  inspection  work,  the 

experiment  should  closely  resemble  the  actual  inspection 

situation  in  several  key  respects. 
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5 .  Estimating  Time  for  Inspector  to  Inspect 


An  estimate  on  the  standard  time  that  it  takes  an 
inspector  to  perform  a  visual  inspection  task  is  necessary  to 
establish  the  range  of  time  over  which  an  inspector  can 
productively  be  employed  on  a  task.  In  regards  to  eliciting 
estimates  of  times  required  to  inspect  for  worker  error  the 
NRC  writes: 

Estimation  of  performance  times  poses  a  more  difficult 
problem  (than  human  error  probabilities).  The  best 
approach  is  to  take  measurements  of  actual  or  simulated 
task  times.  A  second-best  approach  is  to  combine 
estimates  of  those  who  perform  the  tasks.  [Ref  14,  p.  6- 
1] 

Nothing  precludes  empirical  measurement  to  derive  the  required 
time  estimates  of  visual  inspection  tasks  except  the  cost  and 
difficulty  of  doing  so.  There  may  be  several  potential  worker 
errors  within  the  visual  scope  of  the  inspector  at  any  one 
time.  Measuring  the  time  an  inspector  takes  on  any  one  visual 
task  is  complicated  by  the  difficulty  of  the  experimenter 
trying  to  track  where  the  inspector’s  eyes  are  fixated.  The 
costs  of  empirical  measurement  of  times  for  visual  inspection 
tasks  is  large  due  to  the  fact  that  each  facility  must  likely 
derive  its  own  time  estimates  due  to  individual  facility 
peculiarities.  Those  peculiarities  involve,  for  instance, 
level  of  training  of  the  inspectors,  special  equipment  that 
inspectors  use,  and  standard  operating  procedures  within  the 
facility . 
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An  alternative  to  empirical  observation  to  derive  time 
estimates  on  inspectors  is  a  modification  of  the  indirect 
numerical  estimation  technique.  The  standard  inspector  time 
on  one  task  may  be  known  through  empirical  observation.  All 
other  tasks  are  compared  to  one  another  on  a  respondent  survey 
form.  The  respondents  make  directional  judgments  between 
pairs  of  tasks  that  appear  on  the  survey  form.  The  judgments 
call  for: 

1 )  Which  of  the  two  visual  inspection  tasks  requires  more 
time  to  accomplish? 

2)  About  how  much  longer  (in  terms  of  seconds)  does  the 
one  task  take  to  inspect  compared  to  the  other  task? 

Through  simple  algebraic  manipulation  estimates  are  derived 

from  the  survey  as  to  the  amount  of  time  taken  to  perform  all 

visual  inspection  tasks.  This  algebraic  manipulation  is  given 

with  sample  data  in  Appendix  C.  These  point  estimates  are 

averaged  with  the  input  of  others  from  the  same  facility  to 

yield  a  facility-wide  estimate  of  time  for  an  inspector  to 

visually  inspect  every  task. 

D.  THE  COSTS  OF  ERROR 

Many  costs  vie  for  the  attention  of  the  analyst  and  the 
executive.  Swain  writes; 

Most  consideration  must  be  given  to  those  potential  errors 
with  an  intolerable  combined  possibility  of  occurring, 
going  undetected,  and  causing  an  unacceptable  system 
consequence.  [Ref  10,  p.90] 
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An  error  that  causes  unacceptable  system  consequence  is  an 
error  which  is  potentially  too  costly  in  terms  of  the 
consequences  of  interest  for  the  system.  Swain  writes  of  the 
difference  between  errors  that  cause  acceptable  system 
consequence  and  those  that  cause  unacceptable  system 
consequence : 

For  some  errors,  the  system  consequences  may  not  be  severe 
and  perhaps  a  somewhat  high  possibility  of  an  uncaught 
error  can  be  tolerated.  Other  potential  errors  might  have 
such  severe  consequences  that  uncaught  error  probabilities 
of  even  one  in  a  million  could  not  be  tolerated.  For 
example,  a  pharmaceutical  company  could  probably  not 
tolerate  the  existence  of  one  cyanide  capsule  in  one 
million  multivitamin  capsules  delivered  to  the  public. 
[Ref  10,  p. 89] 

The  analyst  must  develop  ways  to  assess  potential  costs  of 
errors  so  that  the  manager  can  allocate  error  catching 
resources  with  regard  to  the  potentially  most  damaging  errors. 

The  reallocation  of  error-catching  resources  in  an 

organization  is  a  change  in  system  design.  It  is  important 

to  remember  that  in  a  systems  approach: 

...successful  design  changes  may  not  always  reduce  the 
probability  of  an  error-but  they  will  reduce  the 
probability  of  undesirable  system  effects  due  to  such 
errors.  [Ref  10,  p  90] 

In  the  multivitamin  example  mentioned  by  Swain,  a  very 
successful  design  change  may  be  one  that  increases  the 
incidence  of  cracked  capsules  (a  frequent  error  of  small 
costs)  in  bottles,  but  absolutely  assures  that  a  cyanide 
capsule  (a  rare  error  of  monumental  costs)  never  shows  up  in 


39 


a  bottle  of  multivitamin  capsules.  It  is  essential  that  the 
potential  costs  of  errors  be  estimated  before  resources  can 
be  allocated  to  reduce  the  probability  of  undesirable  system 
effects . 

For  any  one  type  of  error  there  is  a  corresponding 

distribution  of  costs.  The  U.S.  Army  suggests  a  quick  method 

for  establishing  a  cost  distribution  based  on  the  Project 

Evaluating  and  Review  Technique  (PERT): 

One  might  well  consider  the  possible  use  of  subjective 
cost  estimates  baseci  on  PERT... 

Once  it  has  been  decided  to  employ  the  PERT  technique 
to  determine  likely  costs,  it  becomes  necessary  to 
estimate  the  most  optimistic  cost,  the  most  pessimistic 
cost,  and  the  most  likely  cost. . . 

These  concepts  fit  in  rather  well  with  the  idea  of  a 
generalized  Beta  distribution.  An  adequate  statistical 
model  of  the  Beta  density  to  represent  probable  cost  is: 

(2-3)  f  (  c  )  =  k(c  -  A)p  *  (B-c)q,  A<  <  <B 

where : 

f(c)  =  probability  density  function  of  costs 
c  =  cost 

k  =  constant  to  make  area  under  distribution  curve 

=  1 

(2-4)  k  =  1  /  [(B-A)^*1  *  (3  ( p+1 ,  q+1 )  ] 

p  =  complete  Beta  function 
A  =  minimum  cost 
B  =  maximum  cost 

p,q  =  parameters  determining  the  shape  of  the  beta 
distribution  [Ref  33,  p.36-6] 


From  the  PERT  cost  distribution  a  mean  and  standard  deviation 


can  be  easily  computed. 

(2-5)  m  =  estimate  of  mean  cost 
(2-6)  v  =  estimate  of  variance 

where : 

a  =  estimate  of  minimum  cost 

b  =  estimate  of  maximum  cost 

mQ  =  estimate  of  most  likely  cost 
[Ref  33,  p.36-7] 

See  Appendix  B  for  more  details  of  this  procedure. 

The  PERT  cost  estimation  technique  requires  expert  opinion 

to  generate  estimates  for  the  cost  distribution.  Moder-arul 

Phillips  point  to  an  immediate  supervisor  as  one  capable  of 

providing  estimates  for  PERT: 

The  basis  of  PERT  computations ...  depends  on  the  judgment 
of  the  person  in  charge  of  the  activity  in  question.  He 
is  asked  to  call  on  his  general  experience,  and  his 
knowledge  of  the  requirements  of  the  activity  in  question, 
to  consider  the  personnel  and  facilities  available  to  him, 
and  then  to  estimate  the  three  (costs):  optimistic, 
pessimistic,  and  most  likely  (costs).  [Ref  34,  p  204] 


=  (a  +  4*mc  +  b)  /  6 
=  (b  -  a)2  /  36 


E.  GOAL  PROGRAMMING 

1.  Versatility  and  Utility 

Goal  programming  is  a  powerful  mathematical 
programming  technique  which  allows  for  models  that  very 
closely  approximate  problems  in  industry,  business,  and 
government.  The  power  of  goal  programming  is  wrapped  up  in 
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its  ability  to  overcome  human  analytical  weaknesses.  Paulos 
writes  about  one  of  those  weaknesses: 


There's  a  strong  human  tendency  to  want  everything,  and 
to  deny  that  trade-offs  are  usually  necessary. .  .Trade-offs 
between. . .speed  and  thoroughness. . .are  frequently  muddled 
and  covered  with  a  misty  gauze,  and  this  decline  in 
clarity  is  usually  an  added  cost  for  everyone.  [Ref  7, 
p. 131] 

The  decision  maker  can  often  visualize  and  announce  what  he 
wants  to  accomplish  in  several  competing  areas.  The  level  at 
which  the  decision  maker  wants  to  achieve  is  a  goal.  The 
several  goals  that  a  decision  maker  has  are  often  very 
different.  Budnick,  Mojena,  and  Vollman  write  of  the  power 
of  goal  programming: 

Goal  programming. .. allows  for  consideration  of  multiple 
goals.  The  goals  may  or  may  not  be  of  the  same  dimension 
or  unit  of  measurement.  In  addition,  goal  programming 
allows  for  consideration  of  conflicting  goals.  [Ref  6,  p 
352] 


When  it  comes  to  giving  something  up  for  something 
else  the  decision  maker  often  balks.  Bell,  Keeney,  and  Raiffa 
note  the  quandary  in  which  the  decision  maker  finds  himself: 

This  is  what  the  multi-objective  problem  is  all  about: 

the  making  of  vexing  value  trade-offs.  There  is  no  magic 
formula  for  making  these  value  trade-offs.  The  decision 
maker  would,  of  course,  like  to  do  the  best  he  can  in  each 
attribute,  but  it  is  not  possible  to  maximize  several 
things  at  once.  If  we  reduce  unemployment,  it  may  be  at 
the  expense  of  increasing  inflation...  These  problems  are 
pervasive  and  are  at  the  heart  of  many  public  policy 
controversies.  [Ref  35,  p.4] 
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Any  method  that  helps  coax  the  decision  maker  into  making 
tradeoffs  is  a  help.  Goal  programming  is  a  method  for 
attacking  the  multi-objective  problem. 

A  decision  variable  is  established  which  is  the  entity 
over  which  the  decision  maker  has  some  control  or  influence. 
The  decision  variable  is  often  a  resource.  The  amount  of  the 
resource  that  can  be  employed  is  limited  by  constraints  on  the 
system.  Through  some  known  or  hypothesized  relationships 
varying  amounts  of  the  decision  variable  effect  the  levels  of 
goal  achievement.  The  relationships  by  which  those  decision 
variables  affect  the  decision  maker's  goals  are  captured  by 
mathematical  models  in  the  goal  constraints.  An  achievement 
function  mathematically  combines  the  deviations  from  all 
goals.  By  minimizing  or  maximizing  the  achievement  function 
through  linear  or  nonlinear  programming  methods  a  level  of 
resource  (decision  variable)  use  is  determined  such  that  the 
goals  are  most  closely  achieved. 

Goal  programming  calls  for  the  satisfying  of  goals 
rather  than  the  maximization  of  some  single  objective,  as  is 
often  the  case  in  standard  linear  programming.  The  decision 
maker  states  his  goals,  ranks  them  as  to  their  importance,  and 
weights  them  if  necessary.  The  analyst  determines  the 
relationships  between  a  controllable  variable  and  the 
achievement  of  the  goals.  Goal  programming  gives  the  amount 
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of  the  controllable  variable  to  expend  to  accomplish  as 
closely  as  possible  what  the  decision  maker  desires. 

Budnick,  Mojena,  and  Vollman  explain  the  satisfying 
of  goals  in  this  way: 

With  multiple  goals,  all  goals  usually  cannot  be  realized 
exactly.  Goal  programming  attempts  to  minimize  the 
deviations  from  these  goals  with  consideration  given  to 
the  hierarchy  of  stated  priorities.  [Ref  6,  p  352] 

Ignizio  refers  to  the  greater  applicability  of  multi-objective 

models  to  actual  problems  as  a  reason  to  pursue  goal 

programming.  Hillier  and  Lieberman  present  a  concise  review 

of  goal  programming  [Ref  5,  p  242-252], 

2.  Applied  to  Allocation  of  Inspection  Time 

The  overall  allocation  of  inspection  resources  problem 

is  one  that  is  determined  by  executives  in  an  organization. 

Take,  for  instance,  an  organization  making  an  investment 

decision.  A  million  dollar  budget  for  quality  assurance  is 

being  allocated.  The  decision  of  going  with  twenty  visual 

inspectors  or  two  visual  inspectors  and  robotics  is  a  decision 

far  removed  from  the  floor  manager.  It  is  not  commonly  within 

the  manager's  means  to  hire  or  fire  inspectors  or  buy  new 

inspection-aiding  equipment.  The  manager  must  take  what  the 

organization  provides  him,  whether  it  is  two  or  twenty  visual 

inspectors,  and  make  the  facility  work  to  achieve  quality  and 

quantity  standards.  Basically,  the  only  thing  that  the  floor 
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manager  may  directly  control  in  regard  to  inspection  of  the 
final  product  is  the  allocation  of  time  that  the  inspector 
spends  per  visual  task. 

a.  Decision  Variables 

The  decision  variables  in  goal  programming  are  the 
variables  over  which  the  decision  maker  has  some  control  and 
which  have  some  influence  over  the  achievement  of  the  goals. 
In  goal  programming  to  achieve  allocation  of  visual  inspection 
resources  the  inspector  time  spent  on  checking  the  work  of 
others  is  the  decision  variable  (  Xir  i=  the  number  of  visual 
inspection  tasks).  The  purpose  of  goal  programming  is  to 
determine  an  acceptable  use  of  the  decision  variable. 

b.  System  Constraints 

A  system  is  always  limited  in  the  amount  of  the 
decision  variable  that  cun  be  employed.  This  limitation  is 
due  to  a  limited  supply  of  the  resource  or  limited  capacity 
of  the  system  to  employ  the  resource.  In  goal  programming  to 
allocate  visual  inspection  time  resources  at  least  two  system 
constraints  exist:  the  minimum  and  the  maximum  amounts  of 
time  that  can  be  spent  productively  on  each  of  the  inspection 
tasks.  These  limits  are  based  on  the  experimental  findings 
of  Drury  [Ref  26]  and  Schoonard,  Gould,  and  Miller  [Ref  24], 
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c.  Goal  Constraints 

Goals  reflect  the  objectives  of  the  decision 
maker.  With  visual  inspection  there  is  usually  a  thoroughness 
goal  and  a  speed  goal  that  the  manager  specifies.  These  goals 
are  elastic,  allowing  some  deviation  if  insufficient  resources 
are  available  to  satisfy  both  goals. 

(2-7)  Z  Xi  -  TPOS  +  TNEG  =  TIMEGOAL 

i-1 

where  m  =  the  number  of  inspection  tasks  to  be  done. 
The  right  hand  side  of  this  constraint  represents  a  goal  in 
terms  of  time  in  which  the  manager  wants  inspections  to  be 
completed.  The  deviation  from  the  goal  which  matters  to  the 
manager  is  captured  by  TPOS. 

(2-8)  1  [  f(xa)  *  C(errori)  J  -  ACCPOS  +  ACCNEG  =  ACCGOAL 

1  =  1 

where  f(xi)  =  probability  of  inspector  error  as  a 
function  of  time  he  spends  on  the 
inspection  task, 

C(errori)  =  the  cost  of  erro^  in  meters. 

The  right  hand  side  of  this  constraint  represents  a  goal  in 
terms  of  accuracy  that  the  manager  wants  to  be  met.  The 
deviation  from  the  goal  which  matters  to  the  manager  is 
captured  by  ACCPOS.  Deviations  in  one  goal  versus  another  are 
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encouraged  through  weights  that  are  applied  to  the  respective 
deviations  in  the  achievement  function. 

The  desires  of  the  decision  maker  are  often 
preemptive:  one  tier  of  goals  must  be  satisfied  before 
another  tier  can  be  considered.  Hillier  and  Lieberman 
describe  preemptive  goal  programming  and  the  handling  of 
goals : 

Consider  the  case  for  preemptive  goal  programming.  Such 
a  case  arises  when  one  or  more  of  the  goals  clearly  is  far 
more  important  than  the  others.  Thus  the  initial  focus 
should  be  on  achieving  as  closely  as  possible  these  first 
priority  goals.  After  we  find  an  optimal  solution  with 
respect  to  the  first  priority  goals,  we  can  break  any  ties 
for  the  optimal  solution  by  considering  second  priority 
goals.  When  we  deal  with  goals  on  the  same  priority 
level,  our  approach  is  just  like  the  one  described  for 
nonpreemptive  goal  programming.  [Ref  5,  p.245] 

Assume  a  number  of  independent  visual  inspection 
tasks  which  must  be  accomplished  properly  for  an  error-free 
product  to  come  out  of  the  facility.  Assume  that  a  cost,  a 
quantitative  measure  relating  to  loss  of  customer 
satisfaction,  can  be  associated  with  each  error  on  a  product 
that  leaves  the  facility.  If  the  manager  can  state  a  maximum 
tolerance  cost,  within  which  the  confidence  of  the  consumer 
is  maintained,  then  the  building  of  tiers  can  proceed  in  a 
manner  such  as  this: 

TIER  1. 

Any  inspection  task  covering  a  worker  error  which  has  a 
cost  greater  than  the  maximum  tolerance  cost  should  be 
considered  first  to  maintain  the  consumer's  confidence. 
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TIER  2. 


If  any  time  remains  after  the  first  tier  inspection  tasks 
are  allotted  their  time  then  that  remaining  time  can  be 
applied  to  the  second  tier  of  goals.  The  second  tier 
consists  of  any  visual  inspection  tasks  not  addressed  in 
tier  one. 

If  there  is  insufficient  time  to  cover  all  tasks 
in  the  first  tier,  then  goal  programming  is  invoked  to 

determine  the  allocation  of  time  across  those  first  tier 

visual  inspection  tasks.  If  sufficient  time  exists  for  the 
first  tier  tasks  to  receive  maximum  time  then  they  are  given 
their  full  allotment  of  time  and  goal  programming  is  invoked 
to  allocate  time  on  the  visual  tasks  in  the  second  tier, 
d.  Achievement  Function 

The  achievement  function  takes  the  goals  within 
one  tier  of  the  goal  program  and  considers  the  deviations  from 
each  goal  simultaneously.  The  deviations  from  goals  are  given 
weights  by  the  decision  maker  in  accordance  with  their 

relative  importance  to  the  organization. 

In  the  goal  program  for  allocating  visual 

inspection  resources,  the  decision  maker  gives  a  weight  to  the 
time  goal  deviation  and  the  accuracy  goal  deviation.  The 
appropriate  weight  is  multiplied  by  the  deviation  variable  for 
each  goal.  The  two  weighted  values  are  then  summed  to  equal 
a  total  deviation  for  the  problem.  The  achievement  function 
minimizes  this  total  deviation  value.  As  the  goal  program 
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iterates,  it  trades  off  accuracy  for  time  or  time  for  accuracy 
until  a  solution  is  reached  which  best  satisfies  the  two  goals 
in  light  of  their  respective  weights. 

The  standard  achievement  function  for  allocating 
visual  inspection  time  is  given  in  Equation  2-9. 

(2-9)  Minimize  DEVIATION 

where  DEVIATION  =  (TIMEWT  *  TPOS )  +  (ACCWT  *  ACCPOS ) 
e.  The  Formulation 

The  generic  goal  program  to  accomplish  allocation 
of  inspection  time  is  shown  in  Figure  5. 


Achievement  Function 

Minimize : 

DEVIATION  =  (TIMEWT  *  TPOS)  +  (ACCWT  *  ACCPOS). 


Subject  to: 


Goal  Constraints 


Z  X,  -  TPOS  +  TNEG  =  TIMEGOAL 

i-1 


[  f(xj  *  C( error1 )  ]  -  ACCPOS  +  ACCNEG  =  ACCGOAL 

where  m  =  the  number  of  inspection  tasks  to  be  done. 
System  Constraints 

^mini  S  for  i  =  1,  2,...,  m 

X2,  Xm,  TPOS,  TNEG,  ACCPOS,  ACCNEG  £  0. 


Figure  5  -  Goal  Program  Formulation 


49 


F.  SOFTWARE  SUPPORT  OF  METHODOLOGY 


Suggested  software  support  of  the  proposed  methodology 
includes  computer  packages  that  swiftly  perform  a  number  of 
matrix  operations  and  optimization. 

1.  Matrix  Operations 

A  means  to  manage  matrix  operations  is  needed  to 
transform  the  survey  responses  on  worker  error  into  useable 
data  for  the  model.  Some  survey  responses  will  be  invalid  due 
to  internal  inconsistencies.  As  discussed  in  Chapter  II,  B- 
4,  eigenvalues  are  needed  to  determine  the  internal 
consistency  in  a  group  of  judgments  from  a  respondent.  An 
eigenvalue  solver  or  software  which  will  allow  the  analyst  to 
manipulate  matrices  to  derive  eigenvalues  is  necessary  to 
permit  a  timely  decision  of  whether  to  incorporate  various 
respondent  surveys  into  the  model. 

A  Programming  Language  (commonly  known  as  APL )  is  a 
software  product  that  permits  easy  manipulation  of  matrices. 
The  eigenvalue  solver  that  accompanies  most  APL  software 
packages  is  used  on  the  illustrative  problem  of  this  thesis. 
APL  is  used  in  both  Appendix  A  and  Appendix  D  to  derive  the 
eigenvector  from  respondent  surveys  for  the  probabilities  of 
crew  error  and  to  derive  the  maximum  eigenvalue  to  determine 
respondent  consistency. 
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2.  An  Optimizing  Package 

A  linear  or  nonlinear  programming  package  is  needed 
to  optimize  the  objective  function.  There  are  13  decision 
variables  that  simultaneously  need  to  be  considered. 

a.  Nonlinear  Programming  Package 

A  nonlinear  programming  package  will  permit  a 
straight-forward  solution  to  the  model.  Data  which  show  the 
increase  in  inspector  accuracy  as  a  function  of  time  is 
extracted  from  experimental  studies  which  closely  approximates 
the  type  of  visual  task  being  modeled.  A  curve  fit  is  made 
to  that  data  and  a  function  is  generated  that  approximates  the 
curve  fit.  That  function,  if  convex,  can  be  included  directly 
into  the  goal  constraints  for  use  by  goal  programming.  The 
curves  in  all  studies  indicated  in  this  thesis  have  indeed 
been  convex. 

In  the  illustrative  problem  which  accompanies  this 
thesis  the  versatile  and  powerful  Generalized  Algebraic 
Modeling  System  (GAMS)  package  is  used.  GAMS  allows  for 
nonlinear  programming.  An  example  of  the  input  and  output 
which  is  produced  with  GAMS  is  shown  in  Appendix  E. 

b.  Linear  Programming  Package 

A  linear  programming  package  will  permit  a 
solution  to  the  model.  There  is  a  minor  degree  of  difficulty 
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in  that  the  continuous  curve  generated  by  the  curve  fit  in 
nonlinear  programming  must  be  broken  up  and  made  piecewise 
linear  so  that  a  linear  programming  package  can  solve  the 
problem. 

The  remainder  of  this  thesis  is  the  presentation 
of  a  problem  to  illustrate  the  methodology  of  allocating 
visual  inspection  resources  in  a  production  facility  suggested 
in  Chapter  2. 


Ill . 


BACKGROUND  TO  AN  ARTILLERY  PROBLEM 


A.  THE  ARTILLERY  ORGANIZATION 

The  artillery  fire  support  system  consists  of  separate 
components  which  contribute  to  the  goals  of  "rapid  and 
accurate"  fires.  Depicted  in  Figure  6  are  the  components  of 
the  artillery  system. 


OBSERVER  FIRE  DIRECTION  CENTER 


Figure  6  -  The  Sub-system  Components  of  the  Artillery 


The  observer,  a  Forward  Observer  ( FO )  or  a  RADAR  section, 
spots  a  target.  The  Fire  Direction  Center  (FDC)  receives  the 
target  location  information  from  the  observer  and  translates 
that  input  into  orienting  data  for  the  guns.  The  gun  section 
prepares  the  ammunition,  orients  the  weapon,  and  fires  at  the 
observer's  target.  Survey  sections  and  meteorology  sections 
support  the  gun  section  by  giving  topological  surveyed  grid 
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control  and  atmospheric  meteorological  information  input  into 
the  system. 

The  system  components  noted  above  are  grouped  into 
artillery  organizations.  Several  gun  sections  and  one  FDC 
constitute  a  battery  of  artillery,  commanded  by  a  captain. 
Several  batteries  of  artillery,  observers,  and  surveyors  are 
organized  into  a  battalion  of  artillery,  commanded  by  a 
lieutenant  colonel.  Several  battalions  and  meteorologists 
compose  a  brigade  of  artillery,  commanded  by  a  colonel. 

B.  ROLES  OF  THE  PLAYERS. 

The  focus  of  this  thesis  is  the  Gun  Section.  The  Gun 
Section  consists  of  the  members  depicted  in  Table  1. 

TABLE  1.  MEMBERS  OF  A  GUN  SECTION 


position 

rank 

number 

yrs  in 
service 

Chief 

E-6 

1 

5  - 

15 

Gunner 

E-5 

1 

2  - 

10 

Asst  gunner 

E-4 

1 

1  - 

8 

Ammo  handlers 

E-1,4 

3-7 

0  - 

8 

The  ammunition  handlers  prepare  the  shell  that  carries  the 
munitions  to  the  target,  set  the  fuze  that  activates  the  shell 
at  the  appropriate  point  in  its  rajectory,  and  select  the 
propellant  charge  that  propels  the  shell  out  of  the  cannon  and 
towards  the  target  with  the  proper  velocity.  The  assistant 
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gunner  vertically  orients  the  weapon  tube  to  a  designated 
position  so  as  to  obtain  the  trajectory  needed  to  reach  the 
range  of  the  target.  The  gunner  horizontally  orients  the 
weapon  to  a  designated  position  to  place  the  weapon  tube  on 
line  with  the  target.  The  chief  inspects  the  work  of  his 
subordinates  to  insure  accuracy  of  settings  and  then  commands 
the  crew  to  fire  the  weapon. 

The  FDC  computes  six  elements  of  data  that,  when  taken 
together,  constitute  the  weapon  aimpoint.  Those  six  elements 
of  data  are  the  horizontal  orientation  setting,  vertical 
orientation  setting,  shell  type,  fuze  type,  fuze  setting,  and 
amount  of  propellant.  Firing  the  weapon  at  an  aimpoint  other 
than  that  directed  by  FDC  is  a  gun  crew  error  and  is  the 
responsibility  of  the  chief. 

C.  POSSIBLE  ERRORS  THAT  A  GUN  CREW  MAKES 

The  four  errors  associated  with  the  ammunition  handlers 

are : 

1 )  improperly  selecting  the  shell  type, 

2)  improperly  selecting  the  fuze  type, 

3)  improper  setting  of  the  fuze,  and 

4)  improperly  cutting  the  amount  of  propellant. 

The  four  errors  associated  with  the  assistant  gunner  and  the 

vertical  orientation  of  the  weapon  are: 

1 )  improperly  setting  the  pre-designated  four  digit 
elevation  number  on  the  elevation  counter. 
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2)  improperly  leveling  the  pitch  bubble, 

3)  improperly  leveling  the  cross  level  bubble,  and 

4)  failing  to  have  the  proper  two  digit  number  in  the 
piece  correction  window. 

Figure  7  shows  the  location  of  these  four  errors  on  the 
assistant  gunner's  mount  of  a  towed  howitzer.  The  four  errors 
associated  with  the  gunner  and  the  horizontal  orientation  of 
the  weapon  are: 

1)  improperly  setting  the  pre-designated  four  digit 
number  on  the  deflection  counter, 

2)  improperly  leveling  the  sight  mount  bubbles, 

3)  failing  to  have  the  proper  two  digit  number  in  the 
piece  correction  window,  and 

4 )  not  traversing  the  weapon  to  acquire  a  proper 
sight  picture  through  the  panoramic  telescope  onto 
the  aiming  reference  point. 

Figure  7  shows  the  location  of  these  four  errors  on  the 
gunner's  mount  of  a  towed  howitzer. 

In  addition  to  the  above  twelve  errors  that  a  chief  must 
check  prior  to  firing  his  weapon,  there  are  checks  he  must 
make  regarding  the  firing  data  his  crew  receives  from  the  FDC. 
When  firing  into  a  clearly  defined  area  for  a  prolonged  period 
of  time  a  chief  is  given  a  schematic  that  depicts  the  limits 
of  the  area  into  wnich  he  can  safely  fire.  The  schematic  that 
depicts  the  limits  for  the  safe  fire  of  a  weapon  is  called  a 
"safety  T".  The  chief  is  required  to  check  that  the -settings 
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being  made  by  his  crew  are  within  the  horizontal  orientation 
limits,  vertical  orientation  limits,  fuze  setting  limits, 
propellant  limits,  and  ammunition  constraints  imposed  by  the 
schematic.  In  this  way  the  chief  insures  that  crewmen  have 
not  set  grossly  inaccurate  data  due  to  garbled  communication 
from  the  FDC  or  from  fellow  crew  members.  As  a  minimum,  the 
chief  insures  by  means  of  the  "safety  T"  that  the  round  his 
section  fires  will  fall  within  the  limits  of  safe  fire  for 
munitions  impact. 

D.  THE  GUN  CHIEF'S  TIME  ALLOCATION  PROBLEM 

The  artillery's  primary  mission  is  to  support  the  ground 
gaining  maneuver  forces  with  firepower  to  destroy  the  enemy 
or  diminish  his  will  to  fight.  The  artillery  accomplishes 
this  mission  by  providing  fast  and  accurate  fires  when  called 
upon  to  do  so  [Ref. 43  p.13].  Each  component  of  the  artillery 
fire  support  system  is  called  on  to  contribute  to  the  twin 
goals  of  fast  and  accurate  fire. 

General  Otis  states,  "(Artillery  is)  a  widespread  organism 
whose  ultimate  function  is  to  deliver  massive  firepower 
exactly  when  and  where  the  force  commander  wants  it  [Ref  30, 
p  29]."  Fast  and  accurate  artillery  fire  is  what  the  maneuver 
force  demands  and  what  the  artillery  force  must  deliver.  This 
thesis  recognizes  fast  and  accurate  artillery  fires  as  the 
preeminent  consequences  of  interest  (or  measures  of 
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effectiveness)  for  the  entire  artillery  system.  The  final 
component,  the  gun  section,  shoots  the  projectile  at  the 
target.  The  crew  of  men  that  operate  the  gun  is  the  focus  of 
the  illustrative  problem  of  this  thesis. 

The  primary  resources  available  during  fire  missions  that 
directly  influence  gun  crew  firing  accuracy  are  the  gun 
chiefs.  The  commander  employs  the  gun  chiefs  as  his  personal 
quality  assurance  inspectors  [Ref  31,  p.3-5].  These  personnel 
effect  accuracy  by  inspecting  for  gun  crew  errors  prior  to  the 
firing  of  each  round.  There  are  sixteen  distinct,  mutually 
exclusive  errors  noted  previously  that  the  crew  can  make  when 
riring  the  weapon.  The  limited  resource  that  the  inspector 
expends  to  influence  accuracy  is  the  time  that  he  takes  to 
inspect  for  errors.  The  inspector* 's  time  is  limited  because 
artillery  fires  must  not  only  be  accurate,  but  fast. 
Excessive  time  by  the  inspector  detracts  from  promptness  of 
fires . 

Current  U.S.  Army  literature  does  not  specify  a  method  by 
which  an  inspector  is  to  allocate  his  pre-fire  inspection 
time.  It  is  expected  that  a  gun  chief  will  use  his  artillery 
experience,  knowledge  of  his  crew,  and  intuition  to  decide 
where  his  time  will  best  be  spent  between  the  various  error- 
prone  tasks  he  must  cover.  Intuition  is  inadequate  in  complex 
decision  making,  and  this  problem  requires  a  complex  decision 


[Ref  3,  p.17].  Systems  analysis  and  logic  dictate  the 
spending  of  limited  error-stopping  resources  commensurate  with 
the  importance  of  the  error. 

This  illustrative  problem  presents  an  allocation  of 
inspection  time  model  for  use  by  a  commander.  The  model 
quickly  and  efficiently  analyzes  the  importance  of  gun  crew 
errors  for  an  artillery  unit  by  using  subjective  input  from 
unit  leaders  garnered  by  means  of  surveys.  The  model  then 
uses  goal  programming  and  suggests  an  allocation  of  pre-fire 
inspection  time  to  reduce  the  probability  of  undesirable 
consequences  due  to  errors. 

The  alternative  to  allocation  of  resources  by  modeling  is 
strict  intuition.  If  intuition  is  failing  in  a  unit  as  an 
effective  means  of  allocating  time  then  modeling  offers  an 
alternative.  The  alternative  to  modeling  by  means  other  than 
the  quick  techniques  of  the  methodology  in  this  thesis  is  to 
model  using  input  from  empirical  observations.  The  use  of 
empirical  observations  to  generate  error  data  is  not  viable. 

The  error  data  needed  are  functions  of  training, 
experience,  readiness,  and  type  of  cannon  weapon  system  in  the 
unit.  These  factors  vary  considerably  between  units. 
Consequently,  information  gleaned  in  one  unit  will  have  little 
use  in  another  unit.  The  first  three  factors;  training, 
experience,  and  readiness;  can  change  considerably  within  the 
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same  unit  in  a  matter  of  weeks.  Therefore,  information 
gleaned  in  a  unit  may  have  little  use  in  the  same  unit  the 
following  month.  Gathering  of  unit  error  data  through 
empirical  observation  for  every  artillery  unit  at  frequent 
intervals  is  impractical,  if  not  impossible.  The  modeling 
methodology  must  be  as  efficient  as  the  situation  is  dynamic. 
The  methodology  presented  in  this  thesis  is  useable  by 
artillery  units  to  allocate  inspection  time. 

E.  ASSUMPTIONS 

Assumptions  in  this  problem  are  in  two  categories.  One 
set  of  assumptions  isolates  the  effects  of  errors  by  the  crew 
so  that  other  off-setting  or  magnifying  errors  do  not 
interfere  with  the  analysis  of  the  crew.  The  second  set  of 
assumptions  facilitates  modeling. 

1.  Assumptions  to  Isolate  the  Problem 

The  concern  of  this  illustrative  problem  is  the 
difference  between  the  FDC ' s  aimpoint  (the  target  location) 
and  the  gun  section's  aimpoint  (the  expected  location  of  shell 
impact).  That  difference  is  the  net  effect  of  the  error  that 
the  gun  crew  makes  and  the  chief  fails  to  catch.  This  error 
is  measurable  in  terms  of  the  number  of  meters  by  which  the 
shell  is  expected  to  miss  the  target. 

To  isolate  the  gun  section  regarding  its  contribution 
to  fast  and  accurate  artillery  fire,  it  is  necessary  to  make 
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some  assumptions.  For  the  purpose  of  this  thesis  the 
following  simplifying  assumptions  regarding  source  of  firing 
errors  are  made. 

a.  The  other  artillery  system  components  (besides  the 
gun  section)  perform  their  respective  tasks 
related  to  timely  fire  in  the  time  that  is 
expected  of  them. 

b.  The  aimpoint  computed  by  FDC  is  the  actual 
location  of  the  target. 

1 )  No  errors  are  made  among  the  other 
artillery  system  components  (FO,  RADAR, 

FDC,  Survey,  Meteorology). 

2 )  The  gun  crews  do  not  err  when  taking 
measurements  for  the  FDC.  Gun  crews  are 
periodically  asked  by  the  FDC  to  provide 
propellant  temperatures  and  muzzle 
velocities  to  assist  in  a  better  computer 
solution . 

c.  The  aimpoint  of  the  weapon  is  the  precise  impact 
point  of  the  shell. 

1)  Dispersion  is  disregarded.  Artillery 
fire  always  experiences  normal  dispersion 
in  terms  of  range  probable  error  and 
deviation  probable  error  about  the 
targeted  aimpoint. 

2 )  No  errors  are  made  by  the  battery 
leadership  during  the  emplacement  of  the 
cannon  weapon  system.  Prior  to  being  set 
up  to  fire,  the  unit  leadership  emplaces 
the  weapons.  Errors  occur  during  this 
emplacement  which,  if  not  corrected,  will 
affect  the  accuracy  of  the  artillery  to 
follow. 

3 )  No  errors  are  made  when  the  gun  crew 
performs  routine  serviceability  checks  of 
their  weapon.  Prior  to  weapon  firing  the  . 
gun  section  performs  tasks  which  can 
later  affect  accuracy.  These  tasks  deal 
with  such  things  as  sight  alignment. 
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checks  to  ensure  equipment  is  within 
tolerance,  and  emplacement  of  aiming 
reference  points. 

4)  No  errors  occur  in  sub-systems  directly 
related  to  the  gun  section.  Events 
external  to  the  gun  crew  which  may 
negatively  affect  the  accuracy  of  the 
artillery  are  defective  fuzes,  defective 
propellant,  a  round  that  fails  to  seat  in 
the  chamber  when  it  is  rammed,  and  an 
aiming  reference  point  that  is 
inadvertently  moved. 

2.  Assumptions  to  Facilitate  Modeling 

a.  Battery  Homogeneity 

Gun  sections  are  organized  in  batteries  which 
train  together  and  achieve  some  degree  of  homogeneity  under 
their  common  commander.  Among  the  duties  of  the  commander  is 
the  establishing  of  training  standards,  standard  operating 
procedures,  and  proficiency  standards  for  the  unit.  It  can 
be  reasonably  expected  that  standards  are  somewhat  uniform 
across  a  battery.  This  uniformity  allows  for  the  gathering 
of  unit  data  rather  than  individual  section  data.  Within  the 
unit,  use  of  time  and  human  error  rates  are  basically 
homogenous.  Because  of  battery  homogeneity,  data  generated 
in  one  battery  is  of  limited  or  no  use  in  another  battery. 

b.  Competent  Gun  Chiefs 

A  chief  is  capable.  He  is  competent  as  a  quality 
assurance  inspector  of  his  crew's  work.  Gun  chiefs  are 
required  to  go  through  thorough  periodic  testing  to  insure 
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their  aptitude  at  finding  all  possible  crew  errors.  It  is 
assumed  that  a  chief  does  not  lose  his  ability  to  find 
errors.  The  chief's  competence  is  such  that  he  perceives  his 
visual  tasks  as  being  relatively  simple.  Inspection  failures 
are  the  result  of  random  error,  not  incompetence. 

c.  Gun  Crew  Errors  are  Independent  of  Each  Other 
Crew  errors  are  not  totally  independent.  A 

poorly  performing  crewman  is  more  likely  to  set  a  second 
error  than  a  superb  soldier  is  of  setting  his  first.  The 
interaction  between  errors  is  complex  and  unpredictable.  It 
would  be  extremely  difficult  to  model  that  interaction.  The 
assumption  of  error  independence  is  a  rough  approximation  of 
reality  to  facilitate  modeling. 

d.  Type  I  versus  Type  II  Error 

If  the  costs  of  identifying  a  good  product  as 
flawed  (Type  II  error)  is  minimal  compared  to  the  costs  of 
failing  to  identify  a  flawed  product  as  flawed  (Type  I  error) 
then  Type  II  errors  can  be  disregarded  with  little  loss  in 
model  validity.  This  allows  for  simplification  of  the  model 
and,  in  fact,  is  often  the  case  in  quality  assurance 
inspection  work.  In  the  artillery  gun  section  Type  II  error 
is  commonly  considered  insignificant  when  compared  to  Type  I 
error.  No  chief  has  ever  been  ruined  by  being  overcautious 


and  calling  a  mission  flawed  when  it  was  not.  Many  a  chief 
has  been  ruined  by  failing  to  catch  a  flawed  mission. 

e.  Simplicity  of  Visual  Inspection  Tasks 

The  visual  inspection  tasks  required  on  a  gun 
section  are  relatively  simple  to  a  trained  cannoneer.  The 
types  of  errors  to  be  identified  do  not  require  a  detailed 
search,  exceptional  visual  acuity,  or  higher  level  cognitive 
skills . 

f.  Effects  of  Errors  are  Cumulative 

The  errors  a  gun  section  makes  can  often  off-set 
each  other  such  that  the  full  effect  of  an  error  is  not 
realized.  This  model  assumes  that  when  multiple  errors  occur 
on  the  same  mission  that  their  effects  are  cumulative  along 
one  dimension.  Therefore,  this  is  a  "worst-case"  analysis 
type  model. 
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IV.  MODELING  THE  ARTILLERY  PROBLEM 


A.  GENERAL  CHARACTERISTICS  OF  THE  MODEL 

The  model  to  allocate  a  gun  chief's  inspection  time  is  a 
goal  program  which  suggests  an  amount  of  time  a  gun  chief 
should  spend  on  each  of  thirteen  inspection  tasks  prior  to 
every  round  fired  from  his  weapon.  The  suggestion  for  time 
use  is  based  on  the  following  criteria  for  the  tasks: 

1)  The  likelihood  of  crew  error; 

2)  The  likelihood  of  inspector  error; 

3)  The  potential  costs  of  the  fired  error; 

4)  The  commanders'  objectives. 

The  method  by  which  the  data  are  gathered  for  the  model  takes 
into  account  the  characteristics  of  the  artillery  weapon 
system  and  the  level  of  training  of  the  surveyed  unit  being 
analyzed . 

B.  THE  PROBABILITY  OF  GUN  CREW  ERROR 

Each  of  the  16  crew  tasks  which  the  chief  inspects  with 
his  13  visual  inspection  tasks  has  a  distinct  probability  for 
error .  Based  on  the  battery  homogeneity  assumption,  a 
probability  for  crew  error  applies  across  all  the  gun 
sections  of  a  battery.  As  a  result  of  his  firsthand 
knowledge  of  the  phenomenon,  the  gun  chief  has  a  strong 
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intuitive  sense  for  the  relative  frequency  of  errors  on  the 
crew  tasks  that  he  inspects. 

Due  to  the  battery  homogeneity  assumption  and  the  fact 
that  there  are  six  or  eight  gun  sections  to  a  battery,  there 
are  seldom  more  than  eight  gun  chiefs  to  give  expert  judgment 
on  any  crew  error  phenomenon  under  investigation.  The  paired 
comparison  technique  of  estimating  error  is  not  viable  unless 
there  is  a  large  number  of  respondents.  Thus,  the  paired 
comparison  technique  is  not  a  practical  solution  for  the 
illustrative  problem  in  this  thesis. 

The  model  accepts  the  gun  chief  as  an  expert  capable  of 
rendering  judgments  on  relative  frequencies  of  crew  errors. 
The  model  gathers  this  individual  knowledge  by  means  of  a 
survey.  Appendix  G  contains  copies  of  the  surveys 
administered  to  the  gun  chiefs.  The  survey  forces  the  chief 
to  make  enough  comparisons  to  allow  for  an  internal 
consistency  check  on  his  judgments  (Saaty  [Ref  17]).  Those 
judgments  found  to  be  sufficiently  consistent  are  converted 
to  probabilities  using  the  indirect  numerical  estimation 
technique  (Swain  and  Miller  [Ref  1])  and  averaged  together 
(Von  Winterfeldt  [Ref  29]).  The  result  is  a  deterministic 
point  estimate  of  probability  for  each  crew  error  in  the 
battery.  Appendix  G  shows  the  structure  of  the  survey. 
Appendix  C  demonstrates  checking  the  consistency  of 
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individual  surveys.  Appendix  A  gives  an  example  of 
converting  paired  comparison  survey  input  into  probabilities 
and  the  averaging  of  input  from  several  experts. 

C.  THE  PROBABILITY  OF  INSPECTOR  ERROR 

With  13  inspection  tasks  the  gun  chief  checks  the  16 
possible  crew  errors.  Four  of  the  crew  errors,  the  four 
associated  with  the  assistant  gunner,  are  assessed  by  a 
single  check  that  a  chief  makes  using  a  precision  tool  called 
a  gunner's  quadrant.  Within  certain  bounds  inspector  error 
is  a  function  of  time  taken  to  inspect  a  crew  task.  Those 
bounds  for  a  gun  section  are  explored. 

1.  A  Function  of  Time 

For  a  fraction  of  a  second  at  the  start  of  an 
inspection  task  the  inspector  experiences  no  increase  in 
accuracy  performance  due  to  time  required  for  eyes  to  focus 
and  the  starting  of  the  visual  search.  As  the  inspector 
takes  more  time  his  probability  for  successfully  identifying 
an  error,  given  that  an  error  exists,  increases  at  a 
decreasing  rate  (Rizzi,  Buck,  Anderson  [Ref  25]).  This 
increase  in  accuracy  performance  continues  until  the 
inspector  spends  1 . 5  times  the  standard  amount  of  time  he 
would  normally  spend  on  that  visual  task  (Schoonard,  Gould, 
Mi 1 ler  [ Ref  24 ]  )  . 
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The  experimental  findings  of  Schoonard,  Gould,  and 
Miller  are  heavily  incorporated  to  establish  an  upper  bound 
for  the  productive  use  of  time  for  a  gun  chief  performing 
visual  inspection  tasks.  Schoonard,  Gould,  and  Miller's 
study  [Ref  24]  closely  approximates  the  situation  in  which 
gun  chiefs  find  themselves  when  inspecting  for  crew  error. 
The  study  picked  a  technical  task,  the  inspection  of 
integrated  silicon  circuit  chips,  requiring  trained 
inspectors.  Each  inspector  had  a  minimum  of  six  months 
experience  inspecting  for  faulty  chips.  The  actual  visual 
search  required  the  simultaneous  searching  for  six  or  more 
different  types  of  errors  followed  by  a  decision  to  accept  or 
not  accept  the  chip  construction.  The  possible  errors  ranged 
from  the  simple  to  the  complex  in  terms  of  the  difficulty  of 
discerning  them  during  an  inspection.  The  range  of  times 
that  inspectors  took  to  inspect  a  specific  chip  for  errors 
was  2.7  to  5.4  seconds.  In  every  aspect  of  the  study  noted 
above,  Schoonard,  Gould,  and  Miller  come  extremely  close  to 
the  situation  of  a  gun  chief  inspecting  for  a  gun  crew  error. 

The  model  accepts  that  from  the  time  the  chief  starts 
searching  for  errors  until  1.5  times  the  standard  time  spent 
on  that  inspection  task  [Ref  24]  the  chief  experiences  an 
exponentially  increasing  probability  of  detecting  an  error 
[Ref  25].  At  1.5 


times  the  standard  time  spent  on  an 


inspection  task  a  competent  chief,  with  no  distracting 
factors,  detects  100%  of  errors  (Drury  [Ref  26]).  If  given 
more  than  1.5  times  the  standard  time  for  inspection  the 
inspector  experiences  degradation  in  his  performance  (Kochhar 
[Ref  23]  ) . 

Drury  [Ref  26]  and  Smith  and  Barany  [Ref  22  and  32] 
both  develop  a  curve  which  represents  the  inspector's 
probability  for  accurate  inspection  results  as  a  function  of 
time.  The  curve  increases  at  a  decreasing  rate,  forming  a 
convex  curve.  The  curve  is  continuous  from  a  minimum 
time  which  reflects  the  time  it  takes  the  eye  and  brain  to 
start  a  visual  search  to  a  maximum  time  which  reflects  the 
limits  of  human  vigilance  for  simple  inspection  tasks.  The 
maximum  time  is  assumed  to  be  1.5  times  the  standard  time 
spent  by  the  inspector  on  the  task.  The  function  chosen  to 
represent  the  probability  for  inspector  accuracy  curve  is: 

(4-1)  Yl  =  v/(  Tx  )  ,  i  =  1 . 13 

Where  Yi=  probability  for  inspector  accuracy  on  task  i 

Ti=  (  TI  “  TMINi  )  /  (  ^MAXi  “  TMINi  ) 

Tx=  time  chosen  to  spend  on  taskx 
This  curve  approximates  the  curves  that  Drury  [Ref  26,  p  264] 
and  Smith  and  Barany  [Ref  32,  p  5]  found  in  their  work  and 
which  are  depicted  in  Chapter  II. C. 
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2.  Establishing  Standard  Time  Use  on  Visual  Inspection 

Tasks 

To  utilize  the  experimental  findings  that  greatest 
inspector  accuracy  is  achieved  at  1.5  times  the  standard 
amount  of  time  spent  on  a  visual  inspection  task  requires  the 
generation  of  standard  inspection  times  on  all  visual  tasks. 
The  standard  time  used  to  perform  each  visual  inspection  task 
could  be  gathered  by  empirical  observation,  but  this 
illustrative  problem  uses  the  alternative  of  gathering  the 
chief's  knowledge  of  time  utilization  by  means  of  a  survey 
[Ref  14,  p.6-1] . 

The  chief  has  a  strong  sense  for  where  his  time  is 
spent  when  he  inspects  for  error.  The  chief  performs  paired 
comparisons  of  the  amount  of  time  spent  on  various  inspection 
tasks.  From  the  paired  comparison  data  the  model  derives  a 
point  estimate  for  the  time  spent  per  inspection  task  by  the 
gun  chief.  By  averaging  across  all  chiefs  in  a  battery  the 
model  derives  a  battery  estimate  for  chief's  time  spent  per 
inspection  task.  See  Appendix  C  for  details  on  the 
structuring  of  a  time  use  survey  and  of  how  a  time  use 
estimate  is  derived  from  actual  data. 

3.  Total  Time  to  Perform  All  Inspection  Tasks 

A  limit  on  the  total  amount  of  time  an  inspector  is 
permitted  to  perform  all  visual  tasks  may  be  determined  by  a 
commander.  For  general  purposes  of  this  problem,  the  Field 


Artillery  School  standard  of  30  seconds  is  the  amount  of  time 
permitted  chiefs  to  inspect  their  crews  prior  to  fire  [Ref 
39,  p . 3 ] . 

D.  THE  PROBABILITY  OF  FIRING  AN  ERROR 

For  an  error  to  be  fired  the  chief  must  be  inspecting  an 

improperly  performed  task  and  fail  to  identify  the  error. 

The  probability  that  the  crew  performed  a  given  task 

improperly  is  a  point  estimate  derived  by  survey  input.  The 

probability  that  the  chief  fails  to  identify  the  enor  is  a 

function  of  time.  The  model  is  shown  in  Figure  8. 

A  technique  in  nonlinear  programming  for  evaluating  the 

curve  in  Figure  8  is  shown  in  Equation  4-2.  This  technique 

explicitly  models  the  curve  shape  shown  in  Figure  8. 

(4-2)  P:i4Ci  =  [  V(1  -  PCl)2  *  Ti  +  Pci  ],  i  =  l ,  2 - 13; 

where  T1=  (Tx  -  TM1Ni )  /  ( TMAXl  -  TMINi ) ,  see  Equation  (4-1); 

PlltCi  =  PfaccuracyJ  =  probability  for  system  accuracy 

on  task1, 

Pf  errorj  =  1  -  P( accuracy^  =  probability  for  system 

error  on  taskx. 

E.  THE  COST  OF  FIRING  AN  ERROR 

The  costs  for  firing  an  artillery  gun  crew  error  range 
from  the  insignificant  or  the  unnoticeable  to  the  very 
significant.  Some  of  the  more  important  costs  of  a  gun  crew 
error  being  fired  are: 
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1 )  loss  of  maneuver  force  confidence  in  their 
supporting  artillery; 

2)  the  cost  of  the  resupply  effort  to  deliver  more 
ammunition  for  that  which  was  wasted; 

3)  loss  of  destructiveness  due  to  inaccuracy. 

The  most  important  costs  are  directly  related  to  the 
consequences  of  interest  for  the  system.  Timeliness  and 
accuracy  of  fire  are  the  consequences  of  interest  to  the 
artillery  fire  support  system. 

Each  of  the  16  crew  tasks  has  a  different  cost 
distribution  associated  with  it  in  terms  of  inaccuracy  (in 
meters),  should  the  crewman  err  at  performing  his  task.  The 
model  uses  the  PERT  cost  estimation  technique  to  determine  a 
cost  distribution  for  each  error  [Ref  33]  .  Senior  battery- 
personnel  that  are  well  versed  in  Doth  fire  direction  and  gun 
crew  operations  determine  the  largest,  trie  smallest,  and  the 
most  likely  error  of  each  of  the  16  error  types.  Using 
tabular  firing  tables  [Ref  38]  for  the  battery's  weapon  system 
and  assuming  a  range  at  which  targets  are  most  likely  engaged, 
a  Beta  distribution  cost  curve  (in  meters)  is  established  for 
eacn  error  type  [Ref  38  and  Ref  33].  See  Appendix  B  for 
details  of  the  PERT  cost  estimation  technique. 

The  potential  number  of  meters  by  which  a  crew  error 
causes  the  round  to  miss  a  target  is  a  good  indicator  of 
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Probablity  that  worker  makes  no  error  on  task 
Minimum  time  inspector  must  spend  on  task  to 
get  any  return 

Maximum  time  inspector  can  spend  productively 
before  attention  span  is  exceeded 
Amount  of  time  inspector  spends  on  task 
Probablity  of  no  error  on  finished  product 
Probablity  of  error  on  finished  product 


Figure  8  -  Probability  of  Firing  an  Error 


either  the  amount  of  damage  that  can  be  expected  on  the  target 
(small  errors)  or  the  amount  of  maneuver  conrraence  that  may 
be  lost  (large  errors). 

F.  MULTI-OBJECTIVE  CRITERION  OF  THE  COMMANDER 

The  commander  has  objectives  that  he  wants  accomplished. 
Suppose,  for  instance,  that  a  given  commander,  first  and 
foremost,  wants  never  to  lose  the  confidence  of  the  maneuver 
force  he  supports.  Second,  the  commander  wants  to  have 
effects  on  target.  These  hypothetical  but  realistic 
objectives  of  a  commander  are  pursued  in  the  remainder  of  this 
model  to  demonstrate  the  flexibility  and  real-world 
applicability  of  goal  programming. 

1.  Making  a  Goal  out  of  First  Objective 

For  the  purpose  of  this  problem,  assume  that  the  loss 
of  maneuver  confidence  in  their  supporting  artillery  is  the 
most  costly  and  damaging  thing  that  can  happen  to  the  fire 
support  system.  There  is  good  reason  that  this  may  well  be 
true  for  any  artillery  unit.  Loss  of  confidence  results  in 
hesitance  on  the  part  of  maneuver  to  employ  artillery  and  thus 
increases  the  chances  of  losing  combat  engagements  due  to  lack 
of  firepower. 

Confidence  is  difficult  to  quantify  and  place  on  some 
continuous  scale.  Therefore,  a  crew  error  cannot  be  directly 
an  amount  of  confidence  lost  by  the  maneuver  force 
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towards  its  supporting  artillery.  It  is  intuitive  that  a  miss 
by  a  large  number  of  meters  corresponds  to  a  larger  loss  of 
confidence  by  the  maneuver  whereas  a  small  miss  may  be 
mistaken  as  normal  dispersion  and  thus  goes  unnoticed. 
Therefore  potential  number  of  meters  by  which  a  crew  error 
causes  the  round  to  miss  a  target  suffices  as  a  surrogate  for 
the  amount  of  confidence  lost. 

Below  a  certain  distance  threshold,  maneuver 
confidence  is  not  lost  by  artillery  which  misses  its  target. 
Artillery  is  an  area  fire  weapon  and  it  is  expected  by  those 
who  employ  it  that  artillery  will  hit  in  the  general  vicinity 
for  which  it  was  called.  Within  that  vicinity,  or  expected 
area  of  impact,  the  cost  of  crew  error  is  just  the  cost  of 
missing  the  target  and  the  cost  of  wasted  ammunition. 

The  maneuver  confidence  radius  represents  the  maximum 
distance  from  the  target  that  a  round  can  hit  and  still 
maintain  the  confidence  of  the  maneuver  force  being  supported. 
The  commander's  first  goal  is  that  his  rounds  must  hit  within 
the  maneuver  confidence  radius  of  the  targeted  aimpoint.  The 
commander  needs  to  specify  a  maneuver  confidence  radius. 

2.  Making  a  Goal  out  of  the  Second  Objective 

The  commander's  second  objective,  getting  effects  on 
target,  is  clearly  a  function  of  both  the  accuracy  of  the  fire 
and  the  timeliness  of  the  fire.  If  accuracy  is  sought  at  the 


excessive  expense  of  time  then  the  target  will  likely  move  and 
escape  destruction.  If  timeliness  is  sought  at  the  excessive 
expense  of  accuracy  then  the  target  will  escape  destruction 
due  to  rounds  missing  the  aimpoint.  A  trade-off  between  time 
and  accuracy  is  required  to  get  effects  on  the  target.  The 
commander  needs  to  specify  the  accuracy  and  timeliness  goals. 

G.  GOAL  PROGRAM  FORMULATION 

In  the  proposed  multi-objective  problem  there  are  two 
objectives:  avoid  any  loss  of  maneuver  confidence  and  get 
effects  on  target.  One  goal  is  on  the  first  tier:  hit  within 
the  maneuver  confidence  radius  of  the  target.  Associated  with 
the  second  objective  is  a  tier  of  goals.  On  the  second  tier 
is  an  accuracy  goal  and  a  timeliness  goal  that  the  commander 
specifies  based  on  his  weapon  system  and  the  targets  on  which 
his  weapon  systems  will  most  likely  be  employed. 

This  two-tiered  set  of  goals  lends  itself  to  mult: - 
objective,  preemptive  goal  programming.  On  the  first  tier  is 
the  commander's  goal  of  keeping  maneuver  confidence.  Once 
inspector  time  is  allotted  and  this  first  goal  is  achieved 
then  the  second  tier  of  goals  is  considered. 

1.  Decision  Variables 

In  goal  programming  to  achieve  allocation  of  visual 
inspection  resources,  inspector  time  spent  on  checking  the 
work  of  others  is  the  decision  variable  (  Xx,  i=the  number  of 


visual  inspection  tasks).  The  purpose  of  this  goal 
programming  is  to  achieve  a  most  satisfactory  allocation  of 
inspector  time. 

2.  System  Constraints 

The  minimum  and  the  maximum  amounts  of  time  that  can 
be  spent  productively  on  each  of  the  tasks  are  the  two  system 
constraints.  These  limits  are  based  on  the  experimental 
findings  of  Drury  [Ref  26]  and  Schoonard,  Gould,  and  Miller 
[Ref  24] .  Drury  found  a  minimum  amount  of  time  required  for 
an  inspector's  eyes,  brain,  and  nervous  system  to  function  and 
start  a  visual  search  for  error.  Schoonard,  Gould,  and  Miller 
found  a  maximum  amount  of  time  that  an  inspector  spends  at  a 
visual  inspection  task  before  performance  declines  due  to  the 
limits  of  human  vigilance. 

3.  Goal  Constraints 

There  is  not  a  single  deterministic  cost  associated 
with  each  system  error.  There  are  three  costs:  a  maximum,  a 
minimum,  and  a  most  likely  cost.  Using  the  PERT  cost 
estimation  technique  a  Beta  distribution  curve  is  generated 
to  fit  these  costs.  Visual  inspection  tasks  are  related  to 
accuracy  goals  through  their  individual  beta  cost  functions. 

Visual  inspection  tasks  fall  into  consideration  in 
tier  one  it  their  minimum  possible  cost  exceeds  the  maneuver 
confidence  radius.  Visual  tasks  fall  into  consideration  in 


tier  two  if  their  maximum  possible  cost  exceeds  the  maneuver 
confidence  radius.  Tier  three  visual  tasks  have  maximum 
possible  costs  below  the  maneuver  confidence  radius. 

Time  is  allotted  by  tier,  thus  reflecting  the 
commander's  priorities.  The  total  amount  of  time  required  to 
fully  satisfy  (1.5  times  the  standard  time)  all  visual  tasks 
residing  in  a  tier  is  compared  to  the  total  inspector  time 
available.  If  time  required  is  less  than  time  available  then 
all  visual  tasks  in  the  tier  are  allotted  a  full  complement 
of  time  and  the  model  moves  to  the  next  tier  of  tasks.  If 
time  required  to  fully  satisfy  all  visual  tasks  in  a  tier  is 
greater  than  the  time  available  then  the  goal  program  is  run 
to  determine  allocation  of  inspection  time  among  the  visual 
tasks  residing  in  the  tier. 

Tasks  within  tiers  are  handled  with  the  same  goal 
constraints.  These  goal  constraints  are  elastic,  allowing 
some  deviation  if  insufficient  resources  are  available  to 
satisfy  all  goals. 

The  goal  constraints  are: 

(4-3) 

N 

tier  1  £  Xn  -  TPOS  +  TNEG  =  TIMEGOALj 

n  ■  1 

for  all  tasks  n  where  minimum  cost  taskn  >  MCR, 

p 

tier  2  £  >2  "  TPOS  +  TNEG  =  TIMEGOAL-, 

n=  :  r  * 
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for  all  remaining  tasks  (13  -  N)  where  maximum  cost  of 
taskp  >  MCR, 

tier  3  Xq  -  TPOS  +  TNEG  =  TIMEGOAL3 

for  all  remaining  tasks  where  (N  +  p  +  Q)  =13. 

TIMEGOAI^  is  the  overall  goal  for  time  in  which  all  tasks  are 
to  be  completed.  TIMEGOAL2  is  the  amount  of  time  remaining 
from  TIMEGOALjl  after  the  first  tier  tasks  are  satisfied. 
CIMEGOAL3  is  the  amount  of  time  remaining  from  TIMEGOAL^  after 
the  first  and  second  tier  tasks  are  satisfied. 

(4-4) 

tier  1  g[P(errorn)  *  MAXCOSTJ  -  ACCPOS  +  ACCNEG  =  0 
for  all  tasks  n  where  minimum  cost  taskn  >  MCR, 
tier  9  X  [P(  error. )  *  MAXCOSTJ  -  ACCPOS  +  ACCNEG  =  0 

p=i  ^  v 

for  all  remaining  tasks  (13  -  N)  where  maximum  cost  of 
taskn  >  MCR, 

tier  3  |[P(errorq)  *  MEANCOSTq]  -  ACCPOS  +  ACCNEG  =  0 

for  all  remaining  tasks  where  (N  +  P  +  Q)  =  13, 

where  P(errori)  =  the  probability  of  system  error  on  task^ 

from  Equation  4-2, 

MAXCOSTi  =  maximum  cost  of  task^  (see  Appendix  B), 
MEANCOSTj  =  mean  cost  of  taski  (see  Appendix  B). 
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maneuver  confidence  values 


The  right  hand  side  of  the  constraint  represents  a  goal  in 
terms  of  accuracy  (in  meters)  that  the  commander  wants  to  be 
met . 

4.  Achievement  Function 

The  commander  assigns  a  weight  to  the  time  goal 
deviation  and  the  accuracy  goal  deviation  in  the  model.  The 
commander  must  weight  the  deviations  from  each  goal  with 
respect  to  their  relative  importance  to  the  fire  support 
system.  The  purpose  of  the  goal  program  is  to  simultaneously 
minimize  deviations  from  goals  within  tiers. 

The  commander  is  generally  aware  of  the  type  of  combat 
in  which  he  will  be  employed  and  the  type  of  targets  cn  which 
he  will  be  called  to  deliver  his  fires.  Most  targets  of 
opportunity  have  the  potential  to  move  at  some  rate.  That 
rate  (meters  per  second)  provides  the  commander  the  knowledge 
he  needs  to  make  a  trade-off  between  timeliness  of  fire  and 
accuracy  of  fire.  The  commander  expresses  this  trade-off  by 
the  weight  he  assigns  to  the  respective  goal  deviations. 

The  appropriate  weight  is  multiplied  times  the 
deviation  variable  for  each  goal.  The  two  weighted  values  are 
then  summed  to  equal  a  total  deviation  for  the  problem.  The 
achievement  function  minimizes  this  total  deviation  value. 
As  the  goal  program  iterates,  it  trades  off  accuracy  for  time 


or  time  for  accuracy  until  a  solution  is  reached  which  best 
satisfies  the  two  goals  in  light  of  their  respective  weights. 
The  achievement  function  1°:  qiven  in  Equation  4-5. 

(4-5) 

Minimize : 

DEVIATION  =  (TIMEWT  *  TPOS  )  +  (ACCWT  *  ACCPOS ) . 


E  2 


V.  APPLICATION  TO  ACTIVE  ARMY  UNIT 

A.  CURRENT  ALLOCATION  OF  INSPECTOR  TIME 

The  competent  chief  will  attempt  to  at  least  look  at  every 
one  of  the  16  error-prone  tasks  accomplished  by  his  crew.  The 
chief's  underlying  belief  is  that  he  is  a  good  inspector  and 
that  if  he  looks  at  the  possible  error  he  will  likely  catch 
the  error  if  it  is  present. 

The  following  are  the  keys  to  modeling  the  current 
allocation  of  chief's  time  on  the  pre-fire  inspection: 

1 )  The  gun  chief  spends  at  least  the  theoretically 
derived  minimum  time  for  a  visual  inspection  task 
on  every  task. 

2 )  The  gun  chief  spends  additional  time  on  those 
tasks  that  he  knows  are  more  frequently  in  error 
than  others  in  the  attempt  to  catch  as  many  errors 
as  he  can. 

3)  The  chief  does  not  trade  time  for  accuracy.  He 
shoots  within  the  time  required  of  him  (30 
seconds)  and  checks  as  best  he  can  (using  rule  1 
and  2)  during  that  period  of  time. 

The  model  proposed  is  simple,  yet  it  mirrors  the  chief's 
decision  making  process  quite  well.  The  idea  behind  the 
current  use  of  inspector  time  is  summed  up  by  the  one  goal: 
stop  all  errors  within  the  time  constraint  to  shoot.  A  GAMS 
program  which  reflects  these  priorities  and  uses  the 
probabilities  for  error  developed  in  this  thesis  is  shown  in 
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Appendix  E.  The  allotment  of  inspector  time  resulting  from 
this  model  is  shown  in  TABLE  2. 


B.  ALTERNATIVE  ALLOCATION  OF  INSPECTOR  TIME 

The  following  are  tne  keys  to  modeling  the  alternative 
allocation  of  chief's  time  on  the  pre-fire  inspection: 

1 )  Visual  inspection  tasks  covering  crew  errors  with 
minimum  possible  costs  that  exceed  the  maneuver 
confidence  radius  must  be  considered  first  for 
allocation  of  inspection  time.  The  maximum 
possible  costs  of  the  errors  are  considered  when 
the  allocation  of  time  is  made. 

2 )  Visual  inspection  tasks  covering  crew  errors  with 
maximum  possible  costs  that  exceed  the  maneuver 
confidence  radius  are  considered  next  for 
allocation  of  inspection  time.  The  maximum  costs 
of  the  second  tier  errors  are  considered  when  the 
allocation  is  made. 

3)  Visual  inspection  tasks  covering  the  remaining 
errors  are  considered  last  for  allocation  of 
inspection  time.  The  expected  costs  of  the  third 
tier  errors  are  considered  when  the  allocation  is 
made . 

The  alternative  model  incorporates  this  information  before 
allocating  inspector  time.  In  this  example  of  an  alternative 
plan  for  the  use  of  the  inspector's  time  the  commander's  goal 
is  to  stop  costly  errors.  The  value  that  drives  the  goal,  in 
this  instance,  is  maneuver  force  confidence  and  target 
destruction.  The  results  will  be  different  than  the  current 
allocation  of  inspector  time  which  was  driven  by  the 
inspector's  goal  of  stopping  all  errors. 
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Appendix  E  gives  the  GAMS  computer  runs  of  both  the 
current  and  alternative  allocation  of  inspection  time  by  an 
active  army  unit  surveyed  at  Fort  Ord,  California.  TABLE  2 


shows  the  different  allotments  of  inspector  time  per 
inspection  task  that  resulted  from  the  two  model  runs. 


TABLE  2.  CURRENT  VS  ALTERNATIVE  ALLOTMENT  OF  INSPECTOR  TIME 


Inspector  Task 

Current 

Alternative 

Difference 

Allotment 

Allotment 

Col  2  - 

Coll 

G1 

2.24 

2.80 

+ 

.56 

G2 

2.56 

1.05 

- 

1 . 51 

G3 

1.00 

1.00 

0 

G4 

1.60 

1.46 

- 

.14 

AG 

7.21 

8.90 

+ 

1.69 

A1 

1.80 

1.80 

0 

A2 

1.78 

.75 

- 

1.03 

A3 

1.70 

1.27 

- 

.43 

A4 

3.40 

3.40 

0 

SI 

1 . 14 

3.00 

+ 

1.86 

S2 

2.00 

2.00 

0 

S3 

2.15 

3.50 

+ 

1.35 

S4 

1.40 

1.23 

- 

.17 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

Visual  inspection  to  identify  worker  error  often  involves 
trained  inspectors  to  perform  a  task  requiring  technical 
expertise.  Human  accuracy  at  visual  inspection  tasks  is 
generally  a  function  of  the  time  taken  to  inspect.  Inspector 
accuracy  at  positively  identifying  existing  errors  increases 
at  a  decreasing  rate  as  a  function  of  time.  There  are  lower 
and  upper  bounds  on  the  time  axis  creating  a  range  of  time 
over  which  this  increasing  inspector  accuracy  is  a  continuous 
variable.  Those  bounds  have  been  explored  experimentally  and 
are  available  for  use  by  systems  analysts. 

If  the  possible  human  errors  on  a  work  site  can  be 
identified  then  expert  judgments  of  the  personnel  involved  in 
the  actual  operational  work  can  be  used  to  derive  the 
probability  for  worker  error  on  each  task.  In  the  same  way, 
the  standard  time  an  inspector  spends  on  the  inspection  of  the 
task  can  be  derived.  Costs  of  system  failures  resulting  from 
task  error  can  be  estimated  by  the  management. 

The  expected  cost  of  doing  business  can  be  written  as  a 
function  of  inspector  time  spent  on  each  component  task  in  the 
system.  If  the  decision  maker  can  establish  goals  for  the 


system  cost  of  a  product  and  the  acceptable  amount  of  time  for 
an  inspector  to  spend  on  the  product,  then  goal  programming 
provides  a  method  for  the  organization  to  optimize  the  use  of 
inspector  time. 

B.  POSSIBLE  APPLICATIONS  OF  METHODOLOGY 

Small-scale  operations  that  employ  visual  inspectors  to 
check  the  work  of  production  personnel  are  prime  candidates 
for  use  of  this  methodology.  The  small-scale  operation  is 
emphasized  because  it  is  most  likely  one  in  which  a  formal 
system  analysis  has  not  been  conducted  to  evaluate  optimum 
times  for  visual  inspection  or  optimal  rates  by  which 
production  must  occur. 

An  abbreviated  but  effective  systems  analysis  approach  to 
the  proper  utilization  of  inspector  time  requires  only  a  few 
probability  estimations  and  a  few  cost  estimations.  The  fact 
that  the  personnel  on  the  weapon  systems  can  provide  the 
probability  of  error  information  and  the  unit  leaders  can 
provide  the  cost  of  error  information  means  that  individual 
units  can  apply  systems  analysis  to  better  achieve  their 
goals . 

The  pace  at  which  the  product  is  presented  to  the  visual 
inspector  has  everything  to  do  with  whether  the  visual 
inspection  will  have  the  desired  effect  of  the  management  or 
not.  In  this  case  a  little  bit  of  math  goes  a  long  way.  By 


using  eigenvalues  to  establish  validity  of  expert  judgments, 
the  model  considers  only  those  respondents  whose  judgments  are 
consistent.  Some  adding,  multiplying,  and  dividing  achieves 
probabilities  for  each  crew  error.  The  cost  estimation 
technique  uses  a  simple  beta  distribution  to  model  the  cost 
distribution  for  each  error.  The  model  requires  only  three 
rough  estimates  on  the  part  of  a  knowledgeable  manager.  The 
mean  of  the  cost  distribution  is  achieved  by  come  subtraction 
and  division.  The  probability  of  inspector  error  and  the 
final  cost  of  doing  business  is  determined  by  goal 
programming.  The  input  for  the  goal  constraints  are  obtained 
from  the  decision  maker.  The  result  is  a  recommendation,  by 
task,  for  the  use  of  inspector  time  such  that  the  organization 
goals  are  met  as  closely  as  possible. 

C.  AREAS  FOR  FUTURE  RESEARCH 

A  curve  fit  on  some  experimental  results  done  in  the 
1970 's  on  the  relationship  between  time  spent  on  a  visual 
inspection  task  and  the  accuracy  of  the  inspector  at 
positively  identifying  existing  errors  yielded  a  curve  of 

P  (  I A  )  =  /T  , 

where  P(IA)  is  the  probability  of  inspector  accuracy  and 
T  is  the  time  the  inspector  takes  to  inspect. 

This  is  basically  the  curve  used  for  all  visual  inspection 

tasks  in  the  illustrative  problem  used  in  this  thesis.  An 


area  for  future  research  is  classifying  different  curves  with 
different  levels  of  difficulty  of  the  visual  inspection.  It 
appears  from  the  experimental  results  that  with  an  increase 
in  difficulty  in  the  visual  task,  the  rate  of  improvement  in 
accuracy  changes  from  increasing  at  a  decreasing  rate  to 
increasing  at  a  slower,  more  constant  rate.  If  error-prone 
visual  inspection  tasks  can  be  rated  according  to  difficulty 
and  different  accuracy  curves  can  be  generated  for  difficulty 
levels  then  the  model  may  be  more  valid. 

Another  area  that  may  yield  interesting  results  is 
weighing  of  the  various  expert  judgments  using  some  criteria. 
Suggested  criteria  that  are  readily  discernable  from  the 
respondents  are  time  in  service,  time  in  the  job,  rank,  and 
skill  qualification  test  score.  Weighted  judgments  may  yield 
probabilities  that  are  more  reliable  or  verifiable  than  giving 
each  respondent  equal  weight  in  their  input  to  unit  error 
probabilities.  It  may  be  determined  that  certain 
qualifications  are  significantly  more  important  than  others 
in  the  development  of  a  gun  chief. 

The  model  developed  is  entirely  deterministic.  The  model 
develops  probability  and  time  estimates  from  several  sources. 
From  the  several  estimates  a  distribution  could  be  established 
with  a  standard  deviation  on  the  mean  point  estimate.  The 
cost  estimate  procedure  using  PERT  methods  generates  a  Beta 


distribution  for  costs  of  each  error.  A  simulation  could  be 


developed  incorporating  the  distributions  from  all  the  data 
collected  in  this  model  which  would  yield  a  more  robust 
solution.  By  running  the  simulation  a  number  of  times, 
confidence  intervals  for  the  various  errors  could  be 
generated. 

This  thesis  put  time  and  probability  phenomena  on  an 
interval  scale  of  measurement  by  using  paired  comparisons  and 
indirect  numerical  estimation  methods.  The  anchoring  of  that 
interval  scale  to  a  ratio  scale  was  done  by  direct  numerical 
estimation  of  some  single  phenomenon  by  the  respondents 
Direct  numerical  estimation  has  flaws  as  a  technique  for 
gathering  data  from  operational  personnel.  An  area  for  future 
research  would  be  to  design  an  experiment  and  generate  actual 
values  from  empirical  observation  for  the  estimates  which 
anchor  the  interval  scaled  values  to  a  ratio  scale. 
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APPENDIX  A 


GENERATING  PROBABILITIES  FOR  CREW  ERROR 

The  steps  to  generate  probabilities  for  crew  error  using 
expert  judgment  are  given  below  with  a  numeric  example  taken 
from  survey  input  out  of  an  active  army  unit. 

1.  Create  a  fault  tree.  See  Figure  9.  Worker  errors 
appear  in  the  lowest,  or  first  level.  Errors  are  grouped  by 
worker,  by  work  station,  or  by  some  other  criteria  that  is 
easily  discernable  by  the  inspector.  Those  criteria  form  a 
second  level  of  the  fault  tree  and  fall  under  the  general 
category  of  "error" .  "Error"  and  "no  error"  form  the  third 
level  of  the  tree. 

2.  Gather  survey  responses  from  inspectors.  Inspectors 
are  called  on  to  relay  their  firsthand  knowledge  of  relative 
frequencies  of  errors.  Inspectors  do  pairwise  comparisons  as 
to  relative  frequencies  of  errors  within  criteria  groups. 
When  all  groups  have  been  judged  then  pairwise  comparisons  as 
to  relative  frequencies  are  performed  on  the  groups 
themselves.  Finally  a  direct  numerical  estimation  is  made  by 
the  inspector  as  to  the  frequency  of  missions  that  occur  which 
have  any  error.  A  sample  survey  used  in  an  artillery  unit  is 
included  In  AnppnHiv  n. 
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3.  Turn  survey  responses  into  a  reciprocal  matrix.  Ones 
appear  on  the  main  diagonal.  Survey  responses  of  relative 
frequencies  fill  the  upper  triangular  matrix  and  their 
reciprocals  fill  the  corresponding  positions  in  the  lowe*. 
triangular  matrix.  See  Table  2. 

4.  Derive  the  maximum  eigenvalue  and  the  eigenvector  for 
the  reciprocal  matrix.  The  eigenvector  is  the  relative 


frequency 

of 

each  error 

from  the 

perspective 

of 

one 

respondent . 

See  Table  3. 

The  maximum 

eigenvalue  is 

used  in 

Appendix  C 

to 

derive  a 

measure  of 

consistency 

for 

the 

respondent . 

5 .  Turn 

frequency 

scores  into 

probabilities . 

By 

normalizing 

the 

eigenvector 

(making  its  components  sum  to 

one ) 

the  probability  for  each  error  is  derived,  given  that  some 
error  within  that  group  of  errors  did  occur.  See  Table  4. 

6.  Across  all  consistent  responses  within  a  unit  (use 

method  in  Appendix  C  to  establish  consistency ) ,  average  the 
probability  of  each  error  type  to  make  a  unit  probability. 
See  Table  5.  Fill  those  unit  average  probabilities  back  on 
the  fault  tree  in  the  P(j),  P(k),  P(l),  P(m),  and  P(n) 

positions.  See  Figure  10. 

7.  Take  the  direct  numerical  estimates  of  the  probability 
for  a  fire  mission  containing  an  error  of  any  sort.  Average 


them  for  a  unit  probability  of  fire  mission  error.  Fill  this 
value  in  on  the  fault  tree  in  the  P(ix)  position. 

8.  Use  the  multiplicative  rule  down  each  branch  of  the 
fault  tree  to  derive  a  probability  for  each  type  of  error  on 
the  first  level. 


TABLE  3.  RECIPROCAL  MATRIX  FOR  GUNNER  ERROR  FREQUENCY. 
(  taken  from  a  respondent's  survey  on  gunner  errors  ) 
relative  I 


frequency 
of  errors 

1 

- error 

2 

type--- 

3 

4 

error  1 

1.00 

1.00 

3.00 

3.00 

type  2 

1.00 

1.00 

2.00 

2.00 

3 

0.33 

0.50 

1.00 

1.00 

4 

0.33 

0.50 

1.00 

1.00 

TABLE  4.  EIGENVECTOR  FOR  THE  RECIPROCAL  MATRIX 

error  type  1  .718 
error  type  2  .587 
error  type  3  .263 
error  type  4  .263 


TAELE  5.  NORMALIZED  EIGENVECTOR 
Eigenvector  normalized  to  make  probabilities: 


error 

type 

1 

.39 

error 

type 

2 

.33 

error 

type 

3 

.14 

error 

type 

4 

.14 
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TABLE 


AVERAGE  OF  RESPONDENT'S  ESTIMATES 

Average  of  five  consistent 
respondents'  probabilities. 

error  type  1  .30 

error  type  2  .36 

error  type  3  .18 

error  type  4  .16 
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Figure  10  -  Probability  of  Crew  Error  Data 
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APPENDIX  B. 


GENERATING  COST  ESTIMATES  FOR  ERROR 
Crews  make  errors  and  inspectors  fail  to  identify  those 
errors.  The  costs  that  result  are  measured  in  meters  by  which 
the  error  is  expected  to  cause  the  round  to  miss  the  target. 
Each  error  has  a  maximum  amount  that  it  can  cost  the  system. 
That  maximum  is  affecter  by  such  things  as  unit  standard 
operating  procedures  and  unit  safety  measures  which  may 
preclude  an  extreme  error.  One  example  is  the  M102  howitzer. 
When  traversed,  the  entire  metal  structure  pivots  on  a  ball 
mounted  to  a  base  plate.  If  traverse  limits  exist  for  firing 
safely  then  it  is  a  common  technique  to  drive  metal  stakes  in 
the  ground  abutting  the  metal  structure  when  it  is  lined  up 
to  fire  on  the  left  and  right  traverse  limits  thus  physically 
limiting  the  weapon  to  firing  within  safe  traverse  limits. 
The  result  is  an  administrative  method  of  precluding  gross 
firing  errors  to  the  left  or  right  of  the  target  area.  This 
naturally  reduces  the  possible  cost  of  traverse,  or  deflection 
error . 

Because  there  are  many  procedures  in  the  field  on  five 
different  cannon  weapon  systems  for  precluding  various  firing 
errors,  there  exist  no  minimum  and  maximum  costs  for  crew 
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errors  that  apply  across  the  army.  It  is  within  the  means  of 
senior  personnel  in  each  unit,  however,  to  consider  the  unit's 
procedures  and  weapon  system  to  establish  a  minimum,  maximum, 
and  modal  value  for  each  error  type.  The  steps  to  take  in 
establishing  the  key  "  alues  for  each  error  type  are  reviewed. 

1.  Assume  an  average  range  at  which  targets  are 
engaged.  Assume  a  charge  with  which  those  targets 
are  engaged. 

2.  Extract  information  from  the  tabular  firing  tables 
( TFTs )  for  the  respective  range  and  charge 
assumed.  Information  to  be  extracted  is  the 
effect  (measured  in  meters)  of  a  round  that,  when 
fired,  is  off  by: 

a)  1  mil  in  elevation 

b)  1  mil  in  deflection 

c )  1  square  weight 

d)  1  charge  of  propellant 

e)  0.1  second  on  time  fuze  setting 

3.  Consider  unit  procedures  that  effectively  limit 
the  size  of  errors  that  can  possibly  be  fired. 
Determine  a  maximum  amount  of  error  that  can  be 
set  by  a  worker  in  terms  of  the  items  listed  above 
(mils,  square  weights,  charges,  seconds)  and 
convert  that  amount  to  meters  on  the  ground. 

4.  Given  that  he  must  :  et  an  error,  determine  a 
minimum  amount  of  error  that  a  crewman  can 
possibly  set.  For  example,  a  crewman  can  set  no 
less  than  a  one  charge  error  without  performing 
perfectly  in  that  regard.  So  the  minimum  possiDle 
error  for  charge  is  that  number  of  meters 
corresponding  to  a  one  charge  error. 

5.  Given  that  he  must  set  an  error,  determine  a  m<?st 
likely  error  that  can  be  fired. 
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In  the  case  of  a  105  mm  artillery  unit  at  Fort  Ord, 
California,  the  following  assumptions  were  made  about  the 
potential  costs  of  the  various  inspection  task  errors. 

1)  Safety  Ts  used, 

2)  Safety  stakes  employed, 

3)  Dog-legged  safety  box  in  impact  area, 

4)  Charge  5  propellant  required, 

5)  Range  of  target:  6,000  meters. 

The  following  matrix  was  developed  from  the  expertise  of 
several  unit  leaders. 

TABLE  7.  COST  OF  FIRING  A  CREW  ERROR  (IN  METERS) 

!  estimated  from  I  computed  assuming 


Inspection 

Task 

experience 

a  Beta 

Distribution 

Min 

Cost 

Modal 

Cost 

Max 

Cost 

Mean 

Cost 

Std 

Devn 

G1 

0 

30 

2400 

420 

400 

G2 

0 

10 

200 

40 

33 

G3 

0 

60 

2400 

440 

400 

G4 

0 

15 

400 

74 

66 

AG 

0 

20 

2600 

447 

433 

A 1 

950 

1200 

2400 

1358 

242 

A  2 

25 

30 

100 

41 

12 

A3 

0 

20 

120 

33 

20 

A4 

0 

100 

2500 

483 

417 

SI 

10 

120 

2400 

481 

398 

S2 

15 

130 

2000 

422 

330 

S3 

25 

260 

2500 

594 

412 

S4 

0 

100 

150 

92 

25 
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GENERATING  TIMES  FOR  INSPECTION  TASKS 

Two  methods  are  suggested  by  the  NRC  for  generating 
standard  times  for  inspection  tasks  to  be  completed.  One  is 
by  measuring  empirical  observations  and  the  other  is  by  expert 
judgment.  Making  empirical  observations  is  the  preferred 
method  and  is  the  one  suggested  if  a  unit  has  the  time  and 
resources  to  devote  to  an  experiment  to  measure  a  number  of 
observations . 

The  steps  to  generate  standard  times  for  inspectors  to 
inspect  for  crew  error  (using  expert  judgment)  are  given  below 
with  a  numeric  example  taken  from  survey  input  out  of  an 
active  army  unit. 

1.  Create  a  fault  tree  of  crew  tasks.  See  Figure  10. 
Worker  errors  should  appear  in  the  lowest,  or  first  level. 
Errors  are  grouped  by  worker,  by  work  station,  or  some  other 
criteria  that  is  easily  discernable  by  the  inspector.  Those 
criteria  form  the  second  level  of  the  fault  tree. 

2.  Gather  survey  responses  from  inspectors.  Inspectors 
are  called  on  to  relay  their  firsthand  knowledge  of  relative 
lengths  of  time  required  to  inspect  for  various  errors. 
Inspectors  do  pairwise  comparisons  as  to  relative  lengths  of 
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•+■  i me  to  inspect  for  errors  within  criteria  groups.  When  all 
groups  have  been  judged  then  pairwise  comparisons  as  to 
relative  lengths  of  time  required  to  inspect  the  groups  of 
errors  themselves  are  done.  Finally  a  direct  numerical 
estimation  or  an  empirical  measurement  is  made  by  management 
of  the  time  required  to  inspect  for  one  task. 

3.  Using  the  one  task  as  a  reference  point,  all  other 
tasks  are  given  a  length  of  time  for  inspection.  When  an 
estimate  exists  for  all  tasks  then  a  unit  average  is  taken  and 
that  becomes  the  standard  inspection  time  for  the  unit. 
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CONSISTENCY  CHECKS  ON  SUBJECTIVE  EVALUATIONS 
The  method  for  checking  internal  consistency  of  a  set  of 
pairwise  comparisons  made  by  a  respondent  is  reviewed.  This 
particularly  applies  to  comparisons  of  relative  frequency  of 
various  events,  such  as  event  A  is  three  times  more  likely 
that  event  C. 

1.  Follow  steps  1  through  4  of  Appendix  A  to  derive  a 
maximum  eigenvalue  for  the  respondent's  reciprocal  matrix. 

2.  Generate  a  consistency  index  (Cl)  for  the  respondent's 
matrix  by 

[  >w-N  ]  /  [  N-l  ]  =  Cl. 

3.  generate  a  quantity  of  random  reciprocal  matrices  of  the 
same  dimension  and  containing  the  same  range  of  values  as  was 
possible  to  the  respondents  on  the  survey  they  were 
administered.  The  main  diagonal  of  these  matrices  are  ones. 
The  upper  triangular  cells  of  the  matrices  are  filled  with 
values  picked  at  random  from  the  possible  range.  Complete  the 
lower  triangular  cells  with  the  corresponding  reciprocal 
values  of  the  upper  triangular  cells.  Compute  the  maximum 
eigenvalue  for  each  matrix  and  average  all  of  the  maximum 
eigenvalues  together.  The  result  is  an  estimated  maximum 


eigenvalue  for  a  respondent  who  gives  totally  random  responses 
to  the  original  survey. 

4.  Generate  a  consistency  index  for  the  random  matrix 

^random]  * 

C  ^MAXrandom  -  N]/[N~1]  =  ^random* 

5.  Generate  a  consistency  ratio  by 

(  respondent  )  /  (^^random)  =  CR . 

If  CR  is  less  than  or  equal  to  .10  then  there  is  a  sufficient 
level  of  internal  consistency  in  the  respondent's  judgments 
to  consider  them  for  analysis  work  [Ref  17,  p.83]. 

Step  number  three  above  requires  some  effort  and 
significant  computer  resources  on  the  part  of  an  analyst. 
This  step  is  made  considerably  easier  if  the  analyst  prepares 
the  respondent ' s  survey  so  that  only  values  between  one  and 
nine  (and  their  reciprocals)  can  be  selected  by  the 
respondent.  The  range  of  one  to  nine  recurs  frequently  in 
human  judgments.  For  this  range  of  values  Saaty  [Ref  17] 
generated  consistency  indices  (Cl)  for  random  reciprocal 


matrices  from 

size 

(3  X 

3 )  to 

(10  X 

10). 

Table  7 

shows 

the 

random 

consistency 

indices  for 

matrices  up 

to  (10 

X  10). 

TABLE 

8.  Random 

Consistency 

Indices  (CIrandoin) 

N  X  N 

2 

3 

4 

5 

6 

7 

8 

9 

10 

r  t 

random 

0.00 

0.58 

0.90 

1.12 

1.24 

1.32 

1.41 

1.45  1 

.49 
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USING  GAMS  FOR  GOAL  PROGRAMMING 

GAMS  is  used  to  perform  the  allocation  of  inspection  time 
by  the  Current  Model  and  the  Alternative  Model. 

1 .  The  Current  Model 

In  the  Current  Model  the  chief  is  aware  of  only  the 
inspection  tasks  he  must  perform,  the  relative  frequency  by 
which  the  tasks  are  successful  at  discovering  crew  error,  and 
the  relative  amount  of  time  it  takes  to  perform  the  visual 
tasks.  Given  only  that  much  information  the  chief's  strategy 
is  to  stop  as  many  crew  mistakes  as  he  can  in  the  time 
allowed.  The  assumption  is  made  that  the  chief's  intuition 
works  very  well  with  the  knowledge  that  he  has  and  that  he 
does  accomplish  his  objective  of  minimizing  the  probability 
for  a  crew  mistake.  It  is  reemphasized  that  the  chief  does 
not  consider  the  relative  magnitude  of  the  costs  of  the 
mistakes  when  he  currently  performs  the  pre-fire  inspection. 
The  GAMS  program  for  currently  allocating  inspection  time  is 
labeled  "Current  Model". 

2.  The  Alternative  Model 

The  Alternative  Model  takes  what  the  chief  knows  in 
the  Current  Model  and  then  takes  into  account  the  costs  of  the 
errors.  The  purpose  of  the  Alternative  Model  is  to  not  make 
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costly  mistakes  while  meeting  the  commander's  guidance  for 
fast  and  accurate  fires.  In  the  first  tier  of  the  preemptive 
goal  program  any  inspector  error  which  has  a  minimum  cost 
greater  than  the  commander's  maneuver  confidence  radius  (600 
meters  for  the  illustrative  problem)  is  considered.  Any  first 
tier  error  will  certainly  have  a  huge  negative  effect  on  the 
artillery  system.  Every  possibility  for  first  tier  error  must 
be  precluded.  This  is  done  by  placing  a  large  weight  (BPWT) 
on  any  positive  deviation  from  the  accuracy  goal. 

In  the  second  tier  of  the  preemptive  goal  program  any 
inspector  error  which  has  a  maximum  cost  greater  than  the 
commander’s  maneuver  confidence  radius  is  considered.  Any 
second  tier  error  potentially  has  a  huge  negative  effect  on 
the  artillery  system.  After  first  tier  errors  are  precluded 
then  every  possibility  for  a  second  tier  error  is  precluded. 
Again,  this  is  done  by  placing  a  large  weight  (BPWT)  on  any 
positive  deviation  from  the  accuracy  goal. 

In  the  third  tier  of  the  preemptive  goal  program  all 
remaining  inspector  errors  are  considered.  Time  that  remains 
after  the  first  and  second  tier  errors  are  precluded  is  now 
allotted  to  the  third  tier  inspection  tasks.  The  commander 
now  weights  timeliness  of  fire  and  accuracy  of  fire  relative 
to  each  other.  In  the  illustrative  problem  the  commander 
chooses  to  weight  them  equally  (TIMEWT  =  BPWT  =  1.0). 
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An  asterisk  in  the  first  column  of  the  GAMS  computer 
input  is  a  line  that  the  computer  program  does  not  read.  In 
the  Alternative  Model  (2)  asterisks  are  used  in  the  PARAMETERS 
part  of  the  program  so  that  only  the  inspector  tasks  for  the 
particular  tier  being  considered  are  entered  into  the  program. 
The  GAMS  input  for  the  Alternative  Model's  first  tier  (2)  is 
given.  The  second  tier  and  third  tier  are  the  same  programs 
but  different  tasks  are  allowed  or  disallowed  by  using 
asterisks.  In  each  of  the  three  tiers  the  SCALARS  change  to 
reflect  the  tier.  The  first  tier  starts  with  30  seconds  of 
time  available  (RESPTIME).  The  weights  for  time  deviation 
(TIMEWT)  and  accuracy  deviation  (BPWT)  are  1:600.  The  second 
tier  starts  with  28.2  seconds  of  time  available.  The  weights 
for  time  and  accuracy  deviation  remain  1:600.  The  third  tier 
starts  with  3.6  seconds  of  time  available.  The  weights  for 
time  and  accuracy  deviation  become  1:1. 

3 .  The  Output 

a)  The  Current  Model  output  is  shown  first.  The 
allocation  of  time  by  the  Current  Model  for  each 
inspection  task  is  shown  in  the  column  labeled 
LEVEL  between  the  LOWER  and  UPPER  limits.  This 
appears  near  the  bottom  of  the  page  of  GAMS 
computer  output. 

b)  The  Alternative  Model  output  is  on  three  pages. 

At  the  bottom  of  the  first  page  is  the  allocation 
of  time  for  those  tasks  identified  as  in  the  first 
tier  ( task  A1 ) . 

c)  At  the  bottom  of  the  second  page  is  the  allocation 
of  time  for  those  tasks  identified  as  in  the 
second  tier  (tasks  Gl,  G3,  AG,  A4,  SI,  S2,  S3). 
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d)  At  the  bottom  of  the  third  page  is  the  allocation 
of  time  for  those  tasks  identified  as  in  the  third 
tier  (tasks  G2,  G4,  A2,  A3,  S4 ) . 
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1.  The  Current  Model. 

B  ~ 


GPNLP2 


CURRENT  MODEL 

5  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

4  SET 

5  I  thirteen  inspection  tasks  in  four  categories 

6  /G1,G2,G3,G4,AG,  A1,A2,  A3,  A4 , SI , S2 , S3, S4/ ) 

7  KXXXXXKXXXXXXXXXKXXKXKKXXXXKXXXXXXXXXXXXXXXKXKXKXKKXXXXKXKXKXXXKXXXXXX 

8 

9  PARAMETERS 


10 

11 

XKXXXKXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXKXXXXXXXXXXXXXKXXXXXXXXXXXXXXXXX 

12 

X 

these  min 

times  are  theoretical)  based  on  experiments  of  min  time  for 

13 

X 

eyes  to 

focus  and  mind  to  grasp  what  ayes  are  seeing 

19 

1  s 

M1NU)  min 

time  possible  on  task  i-  theoretical  seconds 

16 

17 

/G1 

.7 

18 

G2 

.7 

19 

G3 

.7 

20 

G4 

.7 

21 

AG 

2.0 

22 

A 1 

.7 

23 

A  2 

.7 

24 

A3 

.7 

25 

A4 

.7 

26 

SI 

.7 

27 

S2 

.7 

28 

S3 

.7 

29 

S4 

.7  / 

30 

31 

XX 

xxxxxxxxxxxxxxxxxxxxwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

32 

X 

these  times 

incorporate  theoretical  1.5  times  max  values  of  standard 

33 

X 

time  taken  by  gun  chiefs  to  perform  the  task.  This  is  time  needed 

34 

X 

to  insure 

(100k)  that  task  is  done  correctly. 

35 

36 

MAX(I)  max 

time  spend  on  task  i  -  from  paired  comparisons 

37 

38 

39 

/G1 

2.8 

40 

G  2 

3.2 

41 

G3 

1.0 

42 

G4 

1.6 

43 

AG 

8.9 

44 

A1 

1.8 

45 

A2 

4.8 

46 

A3 

1.7 

47 

A4 

3.4 

48 

SI 

3.0 

49 

S2 

2.0 

50 

S3 

3.5 

51 

S4 

1.4/ 

52 

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

53 

NEPROBC I ) 

probability  of  no  crew  error  task  i  -  detm  by  INE  method 

54 

55 

/G1 

.992200 

56 

G2 

.990640 

57 

G3 

- 

.995320 

58 

G4 

.995840 

J  08 


The  Current  Model  (continued). 


GPNLP2 


59 

AG 

.979000 

60 

A 1 

.988896 

61 

A2 

.990879 

62 

A3 

.987325 

63 

A9 

.982255 

69 

SI 

.995632 

65 

S2 

.993999 

66 

S3 

.991269 

67 

S9 

.991810/ 

68 

69  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

70  EPROB(I)  probability  of  crew  error  on  task  i  -  detm  by  INE  method 

71 


72 

/G1 

.007800 

73 

G2 

.009360 

79 

G3 

.  009680 

75 

G9 

.009160 

76 

AG 

.026000 

77 

A 1 

.011159 

78 

A2 

.009126 

79 

A3 

.012675 

80 

A9 

.017795 

81 

SI 

.009368 

82 

S2 

.006006 

83 

S3 

.  008736 

89 

S9 

.008190/ 

85 

86  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

87  x  A  chief  does  not  consider  the  costs  of  various  errors  as  he  determines 

88  x  where  he  will  spend  his  time  during  the  inspection.  His  sole  objective 

89  X  is  to  stop  as  many  errors  as  possible  in  the  time  alotted 

90  XXXXXXXXXXXXKXXKKXXXXXXKXXXXXXXXXXXXXXXXKXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

91 

92 

9  3  XXXXXXXXXXXKXXXKXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

99  SCALARS 

95  RESPTIME  total  time  allowed  for  guns  to  shoot  /  30.0/ 

96 

97  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

98  VARIABLES 

°9  X(I)  time  to  be  spent  inspecting  each  task  in  battery 

100  DEVIATION  deviation  from  the  objective  function; 

101 

102  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

103  POSITIVE  VARIABLES 

109  X( I ) ; 

105 

106  KXXXKXXXXXXXXXXXXXXXXXXXXXXKXXXXXXXXKXXKKXXXXXXXKXXXXXXXXXXXXXXXXXXXXK 

107  EQUATIONS 

108  OBJDEF  achievement  function 

109  TLIMIT  time  limit; 

110  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

111  x  this  equation  sums  the  probability  for  system  error  (which  is  a 

112  x  function  of  the  decision  variable)  across  all  the  visual  inspection 

113  x  tasks  AND  assigns  that  sum  a  variable  name  of  DEVIATION. 

119 


The  Current  Model  (continued) 


II! 


GPNLP2 


115  x  MINIMIZE 

116  OBJDEF..  DEVIATION  =E= 

117  (  SUM  (  I, 

118  (  1  -  SORT (  (SQR(EPROB(I)l)  *  ( X( I l-MINC I ) )  /  (MAX( I ) —MIN ( I ) )  ) 

119  -  NEPROB(I)  )  )  )j 

120 

121  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

122  x  -this  constraint  limits  time  spent  by  the  inspector  across  all  visual 

123  X  inspection  tasks  to  less  than  or  equal  to  the  commander's  goal. 

124 

125  x  SUBJECT  TO 

126  TLIMIT . .  SUM  (  I,  X(I)  )  =L=  RESPTIME  ; 

127 

128  X  ADDITIONAL  CONSTRAINTS  involving  min  and  max  times  allowed.  These 

129  x  keep  x  in  the  feasible  region  as  problem  iterates,  otherwise  stops 

130 

131  X.LO(I)  =  MI N( I ) xl . 01 / 

132  X.UP(I)  «  MAX(I); 

133 

134  X  If  region  is  convex,  then  no  matter  where  program  starts,  the 

135  X  solution  will  be  the  same.  This  forces  program  to  start  internally 

136  X  on  the  convex  region  because  funny  things  happen  on  the  edges 

137 

138  X.LCI1  =  MlN< 11X1.02 

139  , 

140  MODEL  GFIILF2  /ALL/; 

141  SOLVE  GFNLP2  USING  NLP  MINIMIZING  DEVIATION; 
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2.  The  Alternative  Model. 


GPNLP2 


4 

5 

6 

7 

8 
9 

10 
11 
12 
1  3 
19 

15 

16 
1  7 
18 

19 

20 
21 
22 
23 
29 

25 

26 

27 

28 

29 

30 

31 

32 

33 
39 

35 

36 

37 

38 

39 
90 


ALTERNATIVE  MODEL  (first  tier) 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
3  x  THIS  MODEL  PERFORMS  FIRST  TIER  OF  PREEMPTIVE  GP  (ON  THOSE  TASKS  WITH 
x  MIN  COST  GREATER  THAN  THE  MANEUVER  CONFIDENCE  RADIUS  =  600m) 


Xresp  time  ballpark 


30 


ti .wt .  bp .  wt 
1  /  600  / 


XXXXKKXXXXXXXXKXXXXXXKXXKKKXXXXXXXXXXXXXKXXXXXXXXXXXKXKXXXXXXXKKXXKXXX 

SET 

I  thirteen  inspection  tasks  in  four  categories 

✓ 

G1 , 

G2  , 

G3, 


G9  , 
AG, 

A 1 

r 

A2, 
A3, 
A  9  , 
SI  , 
S  2 , 
S3 


5  9 


/; 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

PARAMETERS 

xyxxxxxxxxxxxxxxxxxxxxvvvvxxxxxxxxxxyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
x  these  min  times  ere  theoretical;  based  on  experiments  of  min  time-  for 
x  eyes  to  focus  and  mind  to  grasp  what  eyes  Bre  seeing 

MIN(I)  min  time  possible  on  task  i-  theoretical  seconds 


91 

* 

G 1 

.  7 

92 

* 

G2 

.  7 

9  3 

¥ 

G  3 

.  7 

94 

* 

G4 

.  7 

95 

¥ 

AG 

2.0 

46 

A  1 

.  7 

47 

¥ 

A  2 

.  7 

48 

¥ 

A3 

.  7 

49 

¥ 

A4 

.  7 

50 

¥ 

SI 

.7 

51 

¥ 

52 

.  7 

52 

¥ 

S3 

.7 

53 

¥ 

S4 

.7 

59 

55 

56 

57 

58 


xxxxxxxxxxxyxxxxxxxyxxxxxxxxxxKxxxxxxxxxxxxxxyxxxxxxxxxxxxxxxKXXXXxxxx 
X  these  times. incorporate  theoretical  1.5  times  max  values  of  "normal" 
x  time  taken  by  gun  chiefs  to  perform  the  task  This  is  time  needed 
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The  Alternative  Model  (continued). 


GPNIP2 

59  M  to  insura  (100k)  that  task  is  dona  corractly. 

60 

61  MAX(I)  max  time  spend  on  task  i  -  from  paired  comparisons 

62 

63 

64  / 


65 

X 

G1 

2.8 

66 

X 

G2 

3.2 

67 

X 

G3 

1  .  0 

68 

X 

G4 

1  .6 

69 

X 

AG 

8 . 9 

70 

A 1 

1  .8 

71 

X 

A2 

4.8 

72 

X 

A3 

1.7 

73 

X 

A4 

3.4 

74 

X 

SI 

3.0 

75 

X 

S2 

2.0 

76 

X 

S3 

3.5 

77 

X 

S4 

1.4 

78 

/ 

79 

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

80 

NEPR0B( I ) 

probability  of  no  crew  error  task  i  -  detm  by  INE 

81 

82 

/ 

83 

X 

G1 

.992200 

84 

X 

G2 

.990640 

85 

X 

G  3 

.995320 

86 

X 

G4 

.995840 

87 

X 

AG 

.974000 

88 

A 1 

.988846 

89 

X 

A2 

.990874 

90 

X 

A3 

.987325 

91 

X 

A  4 

.982255 

92 

X 

SI 

.995632 

93 

X 

52 

. 993994 

94 

X 

S3 

.991264 

95 

X 

S4 

.991810 

96 

/ 

97 

98 

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

99 

EPROBC I  ) 

probability  of  crew  error  on  task  i  -  detrn  by  IHE 

1  00 

101 

/ 

102 

X 

Cl 

.007800 

103 

X 

G2 

. 009360 

104 

X 

G3 

.004680 

105 

X 

G4 

. 004160 

106 

X 

AG 

.026000 

107 

A 1 

.011154 

103 

X 

A2 

.009126 

109 

X 

A3 

.012675 

110 

X 

A4 

.017745 

111 

X 

SI 

.004368 

112 

X 

S2 

.006006 

113 

X 

S3 

.008736 

114 

X 

S4 

“  .008190 

1  1  2 


The  Alternative  Model  (continued) 


GPNLF2 


115 

116 

117  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

118  x  these  costs  are  derived  using  2  criteria.  First  -  mean  cost  is  based 

119  X  on  data  from  firing  tables  assuming  range  to  target  6000  meters,  and 

120  X  beta  distribution  (pert  estimation  techniques).  Second  -  max  cost 

121  x  is  based  on  data  derived  from  firing  tables. 

122 

123  MCOST(I)  mean  cost  of  error  i  -  detm  by  PERT  Beta  distn  techn 

129 

125 


126 

X 

SET  2  (max  costs) 

127 

/ 

128 

* 

G1 

2900 

1  29 

* 

G2 

200 

1  30 

X 

G  3 

2900 

131 

X 

G9 

900 

132 

X 

AG 

2600  . 

133 

A  1 

2900  . 

1  39 

X 

A2 

100  . 

1  35 

X 

A3 

120  . 

1  36 

X 

A  9 

2500  . 

1  37 

X 

SI 

2900  . 

1  38 

X 

S  2 

2000  . 

1  39 

X 

S3 

2500  . 

190 

X 

S9 

150  . 

191 

/ 

192 

193 

XXXXVXXXXXXKKKXXXKXXXXKXXXXXKKKKXXXXXXXXKXKXXXXXXXXKXXXXXXXXXXXXXXXKXX 

199 

SCALARS 

195 

RESPTIME  total  time  allowed  for  guns  to  shoot  /  30.0/ 

196 

BPARK 

number  meters  in  which  aimpoint  must  be  /  000/ 

197 

TIMEWT 

weight  placed  on  time  pos  devn  in  OBJ  /  1.0/ 

198 

BPI1T 

weight  placed  on  error  pos  devn  in  OBJ  /  600  /; 

199 

150 

xxxvxxyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

1  51 

VARIABLES 

1  52 

X(I  ) 

time  to  be  spent  inspecting  each  task  in  battery 

153 

TOFT 

deviation  from  the  time  goal  constraint 

1  59 

BrOFF 

deviation  from  the  ballpark  goal  constraint 

159 

DEVIATION  devia  +  irr.  *ron  the  objective  function; 

156 

157 

XXXXXXXXXXXXXXyXXXXXXXXXXXKXXXXXXXXXXXXXXXXKXXXXXXXXXXXXXXXXXXXXXXXKXX 

1  55 

POSITIVE 

VARIABLES 

159 

X(I), 

T POS ,  TNEG,  BPPOS ,  BPNEG  ; 

160 

161 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

162 

EQUATIONS 

163 

OBJDEF 

achievement  function 

169 

TLIMIT 

time  limit 

165 

DLIMIT 

distance  limit; 

166 

167 

XVXXXXXXXXXKXXXXXXXXXXXKXXXXXXXXXXXKXXXXXXKXXXXXXXXXXXKXKXXXXKXXXXX 

168 

X 

MINIMIZE 

169 

OBJDEF 

DEVIATION  =  E  = 

170 
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The  Alternative  Model  (continued) 


III 


GPNLP2 


171  C  TIMEWT  *  TPOS  )  +  (  BPHT  *  BPPOS  )| 

172 

173  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

1 7 <i  x  SUBJECT  TO 

175  TllMIT .  .  SUM  (  I.  X(I>  )  -  TPOS  +  TNEG  =  E  =  RESPTIME  ; 

176  DLIMIT. .  SUM  (  I, 

177  (  1  -  SQRT  (  (SQR(EPROBd)  ) )  X  (X( I )-MIN( I ) )  /  (MAX( I )-MIN( I ) )  ) 

178  -  HEPROB(I)  )  x  MCOST(I)  )  -BPPOS  4  BPNEG  =E=  BPARK  ; 

179 

180  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

181  X  ADDITIONAL  CONSTRAINTS  involving  min  end  max  times  allowed.  These 

182  X  keep  x  in  the  feasible  region  as  problem  iterates,  otherwise  stops 

183 

184  X.LO(I)  =  MINCI)xl.0lj 

185  X.UP(I)  =  HAXdli 

186 

187  x  if  region  is  convex,  then  no  matter  where  starts,  soln  should  be  same 

188  x  Need  to  start  internal  baeause  funny  things  happen  on  edges 

189 

190  X . L ( I )  -  MIN(I)X1 .02 

191  t 

192  MODEL  GPULP2  /ALL/; 

193  SOLVE  GPNLP2  USING  NLP  MINIMIZING  DEVIATION; 
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3.  The  Output 

a.  The  Current  Model 


GPNLP2 

SOLUTION  REPORT  SOLVE  GPNLP2  USING  NLP  FROM  LINE  150 


SOLVE 

SUMMARY 

MODEL 

GPNLP2 

OBJECTIVE 

DEVIATION 

TYPE 

NLP 

DIRECTION 

MINIMIZE 

SOLVER 

MIN0S5 

FROM  LINE 

150 

****  SOLVER  STATUS  1  NORMAL  COMPLETION 
****  MODEL  STATUS  2  LOCALLY  OPTIMAL 

****  OBJECTIVE  VALUE  0.0151 


RESOURCE  USAGE, 

LIMIT 

0.263 

1000 . 000 

ITERATION  COUNT, 

LIMIT 

23 

1000 

EVALUATION  ERRORS 

0 

0 

MINOS  - 

—  VERSION 

5.0  APR 

1984 

COURTESY  OF 

B.  A.  MURTAGH 

AND  M.  A. 

SAUNDERS, 

DEPARTMENT  OF  OPERATIONS  RESEARCH, 
STANFORD  UNIVERSITY, 

STANFORD  CALIFORNIA  94305  U.S.A. 


WORK  SPACE  NEEDED  (ESTIMATE) 
WORK  SPACE  AVAILABLE 
(MAXIMUM  OBTAINABLE 


823  WORDS. 
1179  WORDS  . 
164974  WORDS.  ) 


EXIT  --  OPTIMAL  SOLUTION  FOUND 
MAJOR  ITERATIONS  1 

NORM  RG  /  NORM  PI 
TOTAL  USED 
MINOS5  TIME 


5.001 E-09 

0.28  UNITS 

0.17  (INTERPRETER 


0.05) 


LOWER 

LEVEL 

UPPER 

MARGINAL 

- E9U 

OBJDEF 

0 .1300 

0.1300 

0.1300 

-1.0000 

- EQU 

TLIMIT 

-INF 

30.0000 

SO.OOuO 

-0.0022 

OBJPEF 

ACHIEVEMENT  FUNCTION 

TLIMIT 

TIME 

LIMIT 

- VAR 

X 

TIME  TO  BE  SPENT 

INSPECTING  EACH 

task  in  battery 

LOWER 

LEVEL 

UTPER 

MARGINAL 

G1 

0.7070 

2.2427 

2.8000 

EPS 

G2 

0.7070 

2.5660 

3.2000 

EPS 

G3 

0.7070 

1  .0000 

1  .  0000 

-0.0056 

G4 

0.7070 

1.6000 

1.6000 

-0.0001 

AG 

2.0200 

7.2167 

8.9000 

A 1 

0.7070 

1 .8000 

1  .8000 

-0.0029 

A2 

0.7070 

1.7816 

4.8000 

EPS 

A  5 

0.7070 

1.7000 

1  .7000 

-0.0042 

A4 

0.7070 

3.4000 

3.4000 

-0 . 0011 

SI 

0.7070 

1.1417 

3.0000 

EPS 

$2 

0.7070  ’ 

2.0000 

2 .0000 

-0  .  0001 

S3 

0.7070 

2.1513 

■3.5000 

EPS 

S4 

0.7070 

1.4000 

1  ,4000 

-0.0037 

LOWER 

LEVEL 

UPPER 

MARGINAL 

- VAR 

DEVIATION 

-INF 

0.0151 

+  INF 
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b.  The  Alternative  Model's  First  Tier 


GPNLP2 

SOLUTION  REPORT  SOLVE  GPNLP2  USING  NLP  FROM  LINE  193 


SOLVE  SUMMARY 


MODEL  GPMLP2  OBJECTIVE  DEVIATION 

TYPE  NLP  -  DIRECTION  MINIMIZE 

SOLVER  MIN0S5  FROM  LINE  193 

xxxx  SOLVER  STATUS  '  1  NORMAL  COMPLETION 
xxx*  MODEL  STATUS  2  LOCALLY  OPTIMAL 

xxxx  OBJECTIVE  VALUE  O.C 


RESOURCE  USAGE.  LIMIT  0.419  1000  000 

ITERATION  COUNT,  LIMIT  4  1000 

EVALUATION  ERRORS  0  0 


MINOS  —  VERSION  5.0  APR  1984 


COURTESY  OF  B.  A.  MURTAGH  AND  M.  A.  SAUNDERS, 
DEPARTMENT  OF  OPERATIONS  RESEARCH, 
STANFORD  UNIVERSITY, 

STANFORD  CALIFORNIA  94305  U.S.A. 


WORK  SPACE  NEEDED  (ESTIMATE)  —  330  WORDS. 

WORK  SPACE  AVAILABLE  —  454  WORDS. 

(MAXIMUM  OBTAINABLE  —  164974  WORDS.) 


EXIT  --  OPTIMAL  SOLUTION  FOUND 
MAJOR  ITERATIONS  46 

NORM  RG  /  NORM  PI  O.OOOE+OO 
TOTAL  USED  0.43  UNITS 

MINOS5  TIME  0.39  (INTERPRETER 


0.01) 


LOWER 

LEVEL 

UPPER 

MARGINAL 

- EQU 

- EQU 

- EQU 

OBJDEF 

TLIMIT 

DLIMIT 

30 ! 0000 
-26.7696 

30^0000 

-26.7696 

30 ! 00  0  0 
-26.7696 

-1.0000 

EPS 

EPS 

OBJDEF 

TLIMIT 

DLIMIT 

ACHIEVEMENT  FUNCTION 
TIME  LIMIT 

DISTANCE  LIMIT 

- VAR 

X 

TIME  TO  BE  SPENT 

INSPECTING  EACH 

TASK  IN  BATTERY 

LOWER 

LEVEL 

UPPER 

MARGINAL 

A1 

0.7070 

1.8000 

1.8000 

• 

LONER 

LEVEL 

UPPER 

MARGINAL 

- VAR 

- VAR 

- VAR 

- VAR 

- VAR 

DEVIATION 

TPOS 

TNEG 

BPPOS 

BPNEG 

-INF 

28^2000 

4  INF 

♦  INF 

♦  INF 

♦  INF 

♦  INF 

lioooo 

600.0000 

EPS 
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c 


The  Alternative  Model's  Second  Tier 


GPNLF2 

snimin;  REFGR,  SOLVE  GPNLP2  USING  NlP  FROM  LINE  151 


SOLVE  SUMMARY 


MODEL 

GPNLP2 

OBJECTIVE 

DEVIATION 

TYPE 

NLP 

DIRECTION 

MINIMIZE 

SOLVER 

M1NOS5 

FROM  LINE 

191 

XXXX 

SOLVER 

STATUS 

1 

NORMAL  COMPLETION 

XXXX 

MODEL 

STATUS 

2 

LOCALLY  OPTIMAL 

XXXX 

OBJECTIVE  VALUE 

0.0 

RESOURCE  USAGE,  LIMIT 
ITERATION  COUNT,  LIMIT 
EVALUATION  ERRORS 


0.582 

25 

0 


1000.000 

1000 

0 


MINOS  -  VERSION  5.0  APR  1984 


COURTESY  OF  B.  A.  MURTAGH  ANO  M.  A.  SAUNDERS, 
DEPARTMENT  OF  OPERATIONS  RESEARCH, 
STANFORD  UNIVERSITY, 

STANFORD  CALIFORNIA  94305  U.S.A. 


WORK  SPACE  NEEDED  (ESTIMATE) 
WORK  SPACE  AVAILABLE 
(MAXIMUM  OBTAINABLE 


604  HORDS. 
838  IIORDS. 
164974  HORDS . ) 


EXIT 

-- 

OPTIMAL  SOLUTION 

FOUND 

MAJOR  ITERATIONS 

41 

NORM 

RG 

/  NORM  PI 

0  .  OOOE+OO 

TOTAL  USED 

0.60  UNITS 

MINOS5  TIME 

0.51  (INTERPRETER  -  0 

06) 

LOWER 

LEVEL 

UPPER 

— 

EQU 

OBJDEF 

— 

EQU 

TLIMIT 

28^2000 

28 .2000 

28  2000 

— 

EQU 

DLIMIT 

-186.2497 

-186.2497 

-186.2497 

OBJDEF 

ACHIEVEMENT 

FUNCTION 

TLIMIT 

TIME 

LIMIT 

DLIMIT 

DISTANCE  LIMIT 

— 

VAR 

X 

TIME 

TO  BE  SPENT 

INSPECTING  EACH 

TASK  IN  BATTERY 

LOWER 

LEVEL 

UPPER 

MARGINAL 

G1 

0.7070 

2.8000 

2.8000 

EPS 

G3 

0.7070 

1 .0000 

1  .  0000 

EPS 

AG 

2.0200 

8.9000 

8.9000 

. 

A4 

0.7070 

3 .4000 

3.4000 

EPS 

SI 

0.7070 

3.0000 

3.0000 

EPS 

S2 

0.7070 

2.0000 

2 . 0000 

EPS 

S3 

0.7070 

3.5000 

3.5000 

EPS 

LOWER 

LEVEL 

UPPER 

_ 

VAR 

DEVIATION 

-INF 

+  INF 

— 

VAR 

TPOS 

+  INF 

— 

VAR 

TNEG 

3 !  6  000 

+  INF 

— 

VAR 

BFPOS 

+  INF 

— 

VAR 

BPNEG 

+  INF 

MARGINAL 

-1.0000 

EPS 

EPS 


MARGINAL 


i ! oooo 

600!0000 

EPS 
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d.  The  Alternative  Model's  Third  Tier 


GPNLP2 

SOLUTION  REPORT  SOLVE  GPNLP2  USING  NLP  FROM  LINE  208 


SOLVE  SUMMARY 

MODEL  GPHLP2  OBJECTIVE  DEVIATION 

TYPE  NLP  DIRECTION  MINIMIZE 

SOLVER  MIN0S5  FROM  LINE  208 

****  SOLVER  STATUS  1  NORMAL  COMPLETION 
****  MODEL  STATUS  2  LOCALLY  OPTIMAL 

****  OBJECTIVE  VALUE  A. 8104 

RESOURCE  USAGE,  LIMIT  0.309  1000.000 

ITERATION  COUNT,  LIMIT  29  1000 

EVALUATION  ERRORS  0  0 

MINOS  -  VERSION  5.0  APR  1984 


COURTESY  OF  B.  A.  MURTAGH  AND  M.  A.  SAUNDERS, 
DEPARTMENT  OF  OPERATIONS  RESEARCH, 
STANFORD  UNIVERSITY, 

STANFORD  CALIFORNIA  94305  U.S.A. 


WORK  SPACE  NEEDED  (ESTIMATE)  —  511  WORDS. 

WORK  SPACE  AVAILABLE  --  709  HORDS. 

(MAXIMUM  OBTAINABLE  —  164974  WORDS.) 

EXIT  --  OPTIMAL  SOLUTION  FOUND 
MAJOR  ITERATIONS  13 

NORM  RG  ✓  NORM  PI  8.463E-08 
TOTAL  USED  0.32  UNITS 


M 1II0S5 

TINE 

0.25  (INTERPRETER  -  0 

.03) 

LOWER 

LEVEL 

UPPER 

MARGINAL 

- EQU 

OBJDEF 

-1 .0000 

- EQU 

TLIMIT 

3 .6000 

3^6000 

3^6000 

-1 .0000 

- EQU 

DLIMIT 

-7 .1981 

-7 .1981 

-7  . 1981 

-1.0000 

OPJDEF 

ACHIEVEMENT  FUNCTION 

TLIMIT 

TIME 

LIMIT 

DLIMIT 

DISTANCE  LIMIT 

- VAR 

X 

TIME  TO  BE  SPENT 

INSPECTING  EACH 

TASK  IN  BATTERY 

LOWER 

LEVEL 

UPPER 

MARGINAL 

G2 

0.7070 

1.0504 

3.2000 

EPS 

04 

0.7070 

1  .4691 

1.6000 

A2 

0.7070 

0.7508 

4.8000 

EPS 

A3 

0.7070 

1.2784 

1.7000 

EPS 

S4 

0.7070 

1.2390 

1.4000 

EPS 

_ 

LOWER 

LEVEL 

UPPER 

MARGINAL 

- VAR 

DEVIATION 

-INF 

4.8104 

+  INF 

: 

- VAR 

TPOS 

, 

2.1877 

+ 1 NF 

EPS 

- VAR 

TNEG 

, 

+  IIIF 

1.0000 

- VAR 

BPPOS 

2.6227 

+  INF 

. 

- VAR 

BPNEG 

. 

+  INF 

1.0000 
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APPENDIX  F 


GLOSSARY 


A-  Human  Reliability  Terminology 

1.  "Consequence  of  interest  (COI)"  is  analogous  to 
measure  of  effectiveness  (MOE).  MOE  is  used  when 
the  concern  is  maximizing  effectiveness  of  some 
positive  performance  whereas  COI  is  used  when  the 
concern  is  minimizing  consequences  of  some 
negative  performance  (for  example:  human  error). 

2.  "Human  error"  is  any  member  of  a  set  of  human 
actions  that  exceeds  some  limit  of  acceptability. 
The  set  of  actions  must  be  defined  and  the 
tolerance  limits  must  be  established  for  there  to 
be  human  error.  [Ref  10,  p.6] 

3.  "Human  error  probability"  is  the  chance  that  a 
given  task  will  not  be  successfully  completed  by 
personnel  within  a  required  minimum  time  (if  a 
time  requirement  exists).  This  is  the  reciprocal 
of  human  reliability. 

4.  "Human  reliability"  is  the  probability  that  a  task 
will  be  completed  successfully  by  personnel  within 
a  specified  minimum  time  (if  a  time  requirement 
exists).  [Ref  1,  p.  221] 

5.  "Worker"  is  the  human  on  the  production  line  who 
performs  the  error-prone  tasks  that  potentially 
effect  the  finished  product.  A  mistake  by  a 
worker  that  potentially  leads  to  a  flawed  product 
is  referred  to  as  an  "error". 

6.  "Inspector"  is  the  human  tasked  to  find  faulty 
finished  products  before  they  leave  the  production 
line.  A  mistake  by  an  inspector  that  leads  to  a 
flawed  product  is  referred  to  as  a  "failure". 

7.  "System  error"  is  a  worker  error  which  evades 
detection  by  the  inspector  and  has  a  cost  that  is 
of  consequence  of  interest  to  the  system.  A 
system  error  is  also  referred  to  as  a  "flawed 
product " . 
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B.  Artillery  Terminology 

1.  A  "chief"  is  the  senior  man  in  a  crew  of  men  who 
operate  a  gun. 

2.  "Fire(s)",  when  used  as  a  noun,  refers  to  the 
shooting  of  artillery  munitions. 

3.  A  "fire  mission"  is  an  order  from  higher  authority 
to  shoot  one  or  more  rounds  of  artillery  at  a 
target. 

4.  A  "gun"  refers  to  a  single  artillery  cannon  weapon 
system. 

5.  "Gunnery"  refers  to  the  group  of  skills  required 
of  crewmen  to  fire  their  weapon. 

6.  "Maneuver",  when  used  as  a  noun,  refers  to 
infantry  or  armor  elements  to  which  the  artillery 
provides  fire  support. 

7.  A  "round"  is  synonymous  with  the  term  projectile. 

8.  A  "unit"  refers  to  a  battery  of  artillery.  A 
battery  is  either  six  or  eight  cannon  weapon 
systems . 
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APPENDIX  G 


SURVEYS  ADMINISTERED  TO  RESPONDENTS 
The  collection  of  the  expert  judgments  from  operational 
personnel  was  key  to  solving  the  illustrative  problem.  The 
error  probability  survey  and  the  time  use  survey  were  the  two 
surveys  given  to  the  respondents.  Respondents  were  briefed 
in  person  prior  to  working  on  the  survey.  The  surveys  took 
approximately  45  minutes  to  complete. 

The  error  probability  survey  requires  a  gun  chief  to 
compare  pairs  of  errors  as  to  their  relative  likelihood  of 
occurrence.  The  comparisons  are  made  among  similar  type 
errors  within  the  hierarchical  areas  defined  in  Figure  9  in 
Appendix  A.  The  gun  chief  is  also  required  to  make  one  direct 
numerical  estimate  on  the  likelihood  that  his  section  sets  any 
type  of  error.  The  process  by  which  these  survey  results  are 
turned  into  probabilities  is  discussed  in  Appendix  A.  The 
method  by  which  the  respondent's  judgments  are  checked  for 
consistency  is  discussed  in  Appendix  D. 

The  time  use  survey  requires  a  gun  chief  to  compare  the 
amount  of  time  taken  to  perform  one  inspection  task  relative 
another.  Comparisons  are  made  among  similar  type  tasks  within 
hierarchical  areas  defined  in  Figure  9  in  Appendix  A.  One 
inspection  task  which  is  routinely  done  in  a  relatively 
uniform  manner  among  all  gun  chiefs  is  the  inspection  of  the 


121 


gunner's  sight  picture  on  the  gun's  aiming  reference  point. 
It  is  assumed  that  this  task  takes  2.0  seconds  to  perform. 
Using  that  known  value,  all  other  visual  inspection  tasks  are 
scaled  accordingly.  The  process  by  which  the  tasks  are  scaled 
from  the  survey  results  is  discussed  in  Appendix  C. 
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1.  The  Error  Probability  Survey. 


FREQUENCY  BY  WHICH  ERRORS  OCCUR 
(gunner,  asst  gunner,  ammo  handlers,  FDC) 

Which  crew  element  (the  one  on  the  left  or  the  one  on  the 


right)  is  most  likely  to  make  an  error  that  you  must  have 
corrected  prior  to  firing?  How  many  times  more  likely  is  the 
more  error-prone  element  to  make  an  error  than  the  less  error- 
prone  element?  (Assume  daylight  firing,  shell  HE,  fuze  TIME, 
Low  Angle;  with  collimator,  gunner's  quadrant,  and  safety  T. ) 
(  make  one  "X"  per  comparison  below  ) 


WHICH  IS  MORE  LIKELY  ? 


how  many  times 

ERROR  as  likely?  ERROR 

in:  >9  8  7  6  5  4  3  2  2  3  4  5  6  7  8  >9  in: 


. 

gunner 

j  i 

— 

— 

-I-- 

X  ar  | 

— 

— 

— 

— 

— 

— 

asst 

gunner 

gunner 

i  i 

ammo 

handlers 

gunner 

i  i 

-1 

FDC  / 
safety  T 

asst 

gunner 

i  i 

ammo 

handlers 

asst 

gunner 

i  i 

FDC  / 
safety  T 

ammo 

handlers 

i  i 

■  ~  ] 

FDC  / 
safety  T 

Indicate  below  the  number  of  "good"  missions  to  each 
mission  that  you  must  get  involved  in  making  a  correction  on 
either  the  gunner,  asst  gunner,  ammo  handlers,  or  FDC  (a 
"faulty"  mission  is  one  in  which  you  must  get  involved  to  keep 
shooting  accurate.) 


are 


1112 


f  a  :  1 1  y 

mission 


How  many  times  as  likely 
good  missions  to  each  faulty  mission? 

3  4  5  6  7  8  9  10  20  30  40  50  60  >70 


I 

I 

I 


i  I  I  I  i  I  i  I  g°od 

1  I  I  I  1  I  I  Imlssion 
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GUNNER  ERRORS 


Which  error  (the  one  on  the  left  or  the  one  on  the  right)  is 

most  likely  to  occur  (prior  to  your  safety  inspection)  in  your  gun 

section?  How  many  more  times  likely  is  the  more  frequent  event 

than  the  less  frequent  event?  (Assume  daylight  firing,  good 

conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator, 

gunner's  quadrant,  and  safety  T.) 

(  make  one  "X"  per  comparison  below  ) 

(  Mark  only  six  "X"s  on  this  page  ) 

WHICH  IS 
MORE  LIKELY  ? 


how  many  times 

ERROR  as  likely?  ERROR 

in:  >9  8  7  6  5  4  3  2  2  3  4  5  6  7  8  >9  in: 
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ASSISTANT  GUNNER  ERRORS 


Which  error  (the  one  on  the  left  or  the  one  on  the  right)  is 

most  likely  to  occur  (prior  to  your  safety  inspection)  in  your  gun 

section?  How  many  more  times  likely  is  the  more  frequent  event 

than  the  less  frequent  event?  (Assume  daylight  firing,  good 

conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator  and 

gunner's  quadrant.) 

(  make  one  "X"  per  comparison  below  ) 

(  Mark  only  six  "X"s  on  this  page  ) 

WHICH  IS 
MORE  LIKELY  ? 


how  many  times 

ERROR  as  likely?  ERROR 

in:  >9  8  7  6  5  4  3  2  2  3  4  5  6  7  8  >9  in: 
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AMMUNITION  HANDLERS  ERRORS 


Which  error  (the  one  on  the  left  or  the  one  on  the  right)  is 

most  likely  to  occur  (prior  to  your  safety  inspection)  in  your  gun 

section?  How  many  more  times  likely  is  the  more  frequent  event 

than  the  less  frequent  event?  (Assume  daylight  firing,  good 

conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator, 

gunner's  quadrant,  and  safety  T.) 

(  make  one  "X"  per  comparison  below  ) 

(  Mark  only  six  "X"s  on  this  page  ) 

WHICH  IS 
MORE  LIKELY  ? 


hew  many  times 

ERROR  as  likely?  ERROR 

in:  >9  8  7  6  5  4  3  2  2  3  4  5  6  7  8  >9  in: 


FDC  DATA  ERRORS 

(data  is  off  of  Safety  T) 

Which  error  (the  one  on  the  left  or  the  one  on  the  right) 
is  most  likely  to  occur?  How  many  more  times  likely  is  the  more 
frequent  event  than  the  less  frequent  event?  (Assume  daylight 
firing,  good  conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with 
collimator,  gunner's  quadrant,  and  safety  T. ) 

(  make  one  "X"  per  comparison  below  ) 

(  Mark  only  six  "X"s  on  this  page  ) 

WHICH  IS 
MORE  LIKELY  ? 


how  many  times 

ERROR  as  likely?  ERROR 

in:  >9  8  7  6  5  4  3  2  2  3  4  5  6  7  8  >9  in: 
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2.  The  Time  Use  Survey. 

COMPARISON  OF  TIME  TAKEN  TO  PERFORM 
SAFETY  CHECKS  ON  CREW 

Which  check  (the  one  on  the  left  or  the  one  on  the  right) 
consumes  more  of  your  time?  How  many  more  seconds  does  it  take  to 
make  the  longer  check  verses  the  shorter  check  in  each  of  the  six 
comparisons  listed  below?  (Assume  daylight  firing,  good 
conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator, 
gunner's  quadrant,  and  safety  T.  ) 

(  make  one  "X"  per  comparison  below  ) 

WHICH  CHECK  IS  MORE 
TIME  CONSUMING 


increased  number  of  seconds  it  takes  to  do  one  task  vs  the  other 
Usee  >8  7654321  1234567  >8  #sec 
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DEFLECTION  CHECKS 


Which  check  (the  one  on  the  left  or  the  one  on  the  r^ght) 

consumes  more  of  your  time?  How  many  more  seconds  does  it  take  to 

make  the  longer  check  verses  the  shorter  check  in  each  of  the  six 

comparisons  listed  below?  (Assume  daylight  firing,  good 

conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator, 

gunner's  quadrant,  and  safety  T.) 

(  make  one  "X"  per  comparison  below  ) 

(  only  mark  six  "X"s  on  this  form  ) 

WHICH  CHECK  IS  MORE 
TIME  CONSUMING 

increased  number  of  seconds  it  takes  to  do  one  task  vs  the  other 
#sec  >8  7654321  1234567  >8  #sec 


sight 

picture 

! _ 

l  _  _ 

=  = 

bubbles 

sight 

picture 

=  = 

“df 

counter 

sight 

picture 

piece 

corrn 

bubbles 

“df 

counter 

bubbles 

=  = 

piece 

corrn 

df 

counter 

=  = 

piece 

corrn 
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AMMUNITION  CHECKS 


Which  check  (the  one  on  the  left  or  the  one  on  the  right) 
consumes  more  of  your  time?  How  many  more  seconds  does  it  take  to 
make  the  longer  check  verses  the  shorter  check  in  each  of  the  six 
comparisons  listed  below?  (Assume  daylight  firing,  good 
conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator, 
gunner's  quadrant,  and  safety  T. ) 

(  make  one  "X"  per  comparison  below  ) 

WHICH  CHECK  IS  MORE 
TIME  CONSUMING 


increased  number  of  seconds  it  takes  to  do  one  task  vs  the  other 
#sec  >8  7654321  1234567  >8  #sec 


charge 

1  _  „ 

=  = 

time 

setting 

charge 

=  = 

fuze 

type 

charge 

=  = 

shell 

type/wt/lot 

time 

setting 

=  = 

fuze 

type 

time 

setting 

=  = 

i 

shell 

type/wt/lot 

fuze 

type 

—  — 

shell 

type/wt/lot 

FDC  DATA  CHECKS 

(Safety  T  violations) 


Which  check  (the  one  on  the  left  or  the  one  on  the  right) 
consumes  more  of  your  time?  How  many  more  seconds  does  it  take  to 
make  the  longer  check  verses  the  shorter  check  in  each  of  the  six 
comparisons  listed  below?  (Assume  daylight  firing,  good 
conditions,  shell  HE,  fuze  TIME,  Low  Angle;  with  collimator, 
gunner's  quadrant,  and  safety  T. ) 

(  make  one  "X"  per  comparison  below  ) 

WHICH  CHECK  IS  MORE 
TIME  CONSUMING 


increased  number  of  seconds  it  takes  to  do  one  task  vs  the  other 
#sec  >8  7654321  1234567  >8  #sec 
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